Evaluation

Measure model capabilities and application behavior with explicit criteria.

• Benchmarks And Leaderboards (512)
• Eval Harnesses (3)
• Application Evals (2)
• Llm As Judge (1)
• Agent Evals (1)

519 resources · page 11 of 17

• SALAD-Bench

  Creator not supplied; see original source.

  SALAD-Bench GitHub repo: hierarchical safety benchmark of harmful questions plus the MD-Judge evaluator model.

  Not applicable (entity) · unknown · unknown resource · en

• CRMArena

  Creator not supplied; see original source.

  Salesforce CRMArena GitHub repo: benchmarks for LLM agents on CRM tasks (CRMArena and CRMArena-Pro).

  Not applicable (entity) · unknown · unknown resource · en

• Сбер предложил Альянсу в сфере ИИ концепцию независимого бенчмарка (Mobile-Review)

  Creator not supplied; see original source.

  Mobile-review press release in Russian on Sber proposing the MERA benchmark concept to Russia's AI Alliance.

  Not applicable (entity) · unknown · unknown resource · ru

• Scale Labs leaderboards

  Creator not supplied; see original source.

  Scale Labs leaderboards: expert-built benchmarks with held-out sets across coding, reasoning, agents and safety.

  Not applicable (entity) · unknown · unknown resource · en

• Scale SEAL leaderboards

  Creator not supplied; see original source.

  Scale leaderboards page (now Scale Labs): expert-built LLM benchmarks across coding, reasoning and more.

  Not applicable (entity) · unknown · unknown resource · en

• ScanNet benchmark

  Creator not supplied; see original source.

  ScanNet GitHub repo: RGB-D video dataset of 1,500+ indoor scans with 3D annotations and benchmark.

  Not applicable (entity) · unknown · unknown resource · en

• SciCode

  Creator not supplied; see original source.

  SciCode benchmark site: research-level scientific coding problems for evaluating language models.

  Not applicable (entity) · unknown · unknown resource · en

• ScienceAgentBench

  Creator not supplied; see original source.

  ScienceAgentBench project page: data-driven scientific discovery tasks from published papers for evaluating language agents.

  Not applicable (entity) · unknown · unknown resource · en

• ScreenSpot-Pro

  Creator not supplied; see original source.

  ScreenSpot-Pro GitHub repo: GUI grounding benchmark on high-resolution professional desktop screenshots.

  Not applicable (entity) · unknown · unknown resource · en

• SEA-HELM

  Creator not supplied; see original source.

  SEA-HELM GitHub repo from AI Singapore: evaluation framework for LLMs on Southeast Asian languages and culture.

  Not applicable (entity) · unknown · unknown resource · en

• SEA-HELM leaderboard (AI Singapore)

  Creator not supplied; see original source.

  SEA-HELM leaderboard (AI Singapore): holistic evaluation of LLMs on Southeast Asian languages.

  Not applicable (entity) · unknown · unknown resource · my|fil|id|ms|ta|th|vi

• SeaExam / LLM Leaderboard for SEA (SeaLLMs)

  Creator not supplied; see original source.

  SeaLLMs SeaExam leaderboard Hugging Face Space: LLM results on Southeast Asian language exams.

  Not applicable (entity) · unknown · unknown resource · mul

• SEED-TTS-EVAL

  Creator not supplied; see original source.

  ByteDance seed-tts-eval GitHub repo: objective test set and scripts for evaluating TTS models.

  Not applicable (entity) · unknown · unknown resource · en

• Shaastra (IIT-M): "Marking benchmarks"

  Creator not supplied; see original source.

  IIT Madras Shaastra feature on rethinking AI evaluation for real-world deployment, including Indic benchmarks.

  Not applicable (entity) · unknown · unknown resource · en

• SILMA Arabic Broad Leaderboard

  Creator not supplied; see original source.

  SILMA Arabic Broad Leaderboard Hugging Face Space: model results on a broad Arabic benchmark.

  Not applicable (entity) · unknown · unknown resource · ar

• SimpleBench

  Creator not supplied; see original source.

  SimpleBench site: multiple-choice benchmark of questions that ordinary humans answer easily but LLMs often miss, with a human baseline.

  Not applicable (entity) · unknown · unknown resource · en

• SimpleQA (simple-evals)

  Creator not supplied; see original source.

  OpenAI simple-evals GitHub repo: reference implementations of evals such as SimpleQA, HealthBench and BrowseComp.

  Not applicable (entity) · unknown · unknown resource · en

• SimpleQA Verified

  Creator not supplied; see original source.

  SimpleQA Verified paper: a cleaned 1,000-prompt version of SimpleQA for measuring short-form factual recall of LLMs.

  Not applicable (entity) · unknown · unknown resource · en

• SimplerEnv (SIMPLER)

  Creator not supplied; see original source.

  SimplerEnv project page: simulated environments for evaluating real-world robot manipulation policies such as Google Robot and WidowX.

  Not applicable (entity) · unknown · unknown resource · en

• Singapore AISI (Digital Trust Centre)

  Creator not supplied; see original source.

  Singapore AI Safety Institute (Digital Trust Centre): research on AI safety science and governance.

  Not applicable (entity) · unknown · unknown resource · en

• SongBench

  Creator not supplied; see original source.

  Tencent SongBench GitHub repo: benchmark evaluating AI-generated songs across multiple musical dimensions.

  Not applicable (entity) · unknown · unknown resource · en

• SongEval

  Creator not supplied; see original source.

  SongEval GitHub repo: song aesthetic evaluation toolkit and dataset.

  Not applicable (entity) · unknown · unknown resource · en

• Spider 2.0

  Creator not supplied; see original source.

  Spider 2.0 site: 632 enterprise text-to-SQL workflow problems for evaluating language models and agents.

  Not applicable (entity) · unknown · unknown resource · en

• Spiral-Bench (EQ-Bench)

  Creator not supplied; see original source.

  EQ-Bench Spiral-Bench leaderboard, scoring how models handle simulated users drifting into delusional or harmful spirals.

  Not applicable (entity) · unknown · unknown resource · en

• Stanford HAI/RegLab: legal models hallucinate in 1 of 6+ queries

  Creator not supplied; see original source.

  Stanford HAI article on a RegLab study measuring hallucination rates of legal AI research tools such as Lexis+ AI and Westlaw AI.

  Not applicable (entity) · unknown · unknown resource · en

• StrongREJECT

  Creator not supplied; see original source.

  StrongREJECT documentation: benchmark package that scores whether jailbreaks yield useful harmful outputs rather than mere compliance.

  Not applicable (entity) · unknown · unknown resource · en

• SuperCLUE

  Creator not supplied; see original source.

  SuperCLUE: Chinese LLM benchmark with general, reasoning, multimodal, agent and application leaderboards.

  Not applicable (entity) · unknown · unknown resource · zh

• SuperCLUE GitHub

  Creator not supplied; see original source.

  SuperCLUE GitHub repo: Chinese general foundation model benchmark with methodology and results.

  Not applicable (entity) · unknown · unknown resource · zh

• SuperCLUE-Safety

  Creator not supplied; see original source.

  SuperCLUE-Safety GitHub repo: a multi-turn adversarial safety benchmark for Chinese large language models.

  Not applicable (entity) · unknown · unknown resource · zh

• SuperGLUE

  Creator not supplied; see original source.

  SuperGLUE benchmark site: harder successor to GLUE with language-understanding tasks, toolkit and leaderboard.

  Not applicable (entity) · unknown · unknown resource · en

• SWE-bench

  Creator not supplied; see original source.

  SWE-bench leaderboards: coding-agent results on resolving real GitHub issues, across SWE-bench Verified, Multilingual and Multimodal.

  Not applicable (entity) · unknown · unknown resource · en

• SWE-bench Pro

  Creator not supplied; see original source.

  Scale AI SWE-bench Pro GitHub repo: benchmark of long-horizon software engineering tasks for agents.

  Not applicable (entity) · unknown · unknown resource · en

Plain-text Markdown copy