Evaluation & benchmarks for advanced learners
452 resources · page 11 of 15
SALAD-Bench
SALAD-Bench GitHub repo: hierarchical safety benchmark of harmful questions plus the MD-Judge evaluator model.
CRMArena
Salesforce CRMArena GitHub repo: benchmarks for LLM agents on CRM tasks (CRMArena and CRMArena-Pro).
Сбер предложил Альянсу в сфере ИИ концепцию независимого бенчмарка (Mobile-Review)
Mobile-review press release in Russian on Sber proposing the MERA benchmark concept to Russia's AI Alliance.
Scale Labs leaderboards
by Scale AI
Scale Labs leaderboards: expert-built benchmarks with held-out sets across coding, reasoning, agents and safety.
Scale SEAL leaderboards
by Scale AI
Scale leaderboards page (now Scale Labs): expert-built LLM benchmarks across coding, reasoning and more.
ScanNet benchmark
by ScanNet
ScanNet GitHub repo: RGB-D video dataset of 1,500+ indoor scans with 3D annotations and benchmark.
SciCode
SciCode benchmark site: research-level scientific coding problems for evaluating language models.
ScienceAgentBench
by OSU Natural Language Processing
ScienceAgentBench project page: data-driven scientific discovery tasks from published papers for evaluating language agents.
ScreenSpot-Pro
by Kaixin Li
ScreenSpot-Pro GitHub repo: GUI grounding benchmark on high-resolution professional desktop screenshots.
SEA-HELM
by AI Singapore
SEA-HELM GitHub repo from AI Singapore: evaluation framework for LLMs on Southeast Asian languages and culture.
SEA-HELM leaderboard (AI Singapore)
SEA-HELM leaderboard (AI Singapore): holistic evaluation of LLMs on Southeast Asian languages.
SeaExam / LLM Leaderboard for SEA (SeaLLMs)
by SeaLLMs
SeaLLMs SeaExam leaderboard Hugging Face Space: LLM results on Southeast Asian language exams.
SEED-TTS-EVAL
ByteDance seed-tts-eval GitHub repo: objective test set and scripts for evaluating TTS models.
Shaastra (IIT-M): "Marking benchmarks"
IIT Madras Shaastra feature on rethinking AI evaluation for real-world deployment, including Indic benchmarks.
SILMA Arabic Broad Leaderboard
by SILMA AI
SILMA Arabic Broad Leaderboard Hugging Face Space: model results on a broad Arabic benchmark.
SimpleBench
by SimpleBench (Philip, AI Explained)
SimpleBench site: multiple-choice benchmark of questions that ordinary humans answer easily but LLMs often miss, with a human baseline.
SimpleQA (simple-evals)
by OpenAI
OpenAI simple-evals GitHub repo: reference implementations of evals such as SimpleQA, HealthBench and BrowseComp.
SimpleQA Verified
SimpleQA Verified paper: a cleaned 1,000-prompt version of SimpleQA for measuring short-form factual recall of LLMs.
SimplerEnv (SIMPLER)
by simple-env
SimplerEnv project page: simulated environments for evaluating real-world robot manipulation policies such as Google Robot and WidowX.
Singapore AISI (Digital Trust Centre)
by Singapore AI Safety Institute
Singapore AI Safety Institute (Digital Trust Centre): research on AI safety science and governance.
SongBench
by Tencent
Tencent SongBench GitHub repo: benchmark evaluating AI-generated songs across multiple musical dimensions.
SongEval
by ASLP-lab
SongEval GitHub repo: song aesthetic evaluation toolkit and dataset.
Spider 2.0
by spider2-sql
Spider 2.0 site: 632 enterprise text-to-SQL workflow problems for evaluating language models and agents.
Spiral-Bench (EQ-Bench)
EQ-Bench Spiral-Bench leaderboard, scoring how models handle simulated users drifting into delusional or harmful spirals.
Stanford HAI/RegLab: legal models hallucinate in 1 of 6+ queries
by Stanford HAI
Stanford HAI article on a RegLab study measuring hallucination rates of legal AI research tools such as Lexis+ AI and Westlaw AI.
StrongREJECT
by StrongREJECT team (UC Berkeley)
StrongREJECT documentation: benchmark package that scores whether jailbreaks yield useful harmful outputs rather than mere compliance.
SuperCLUE
by SuperCLUE (CLUE benchmark team)
SuperCLUE: Chinese LLM benchmark with general, reasoning, multimodal, agent and application leaderboards.
SuperCLUE GitHub
SuperCLUE GitHub repo: Chinese general foundation model benchmark with methodology and results.
SuperCLUE-Safety
SuperCLUE-Safety GitHub repo: a multi-turn adversarial safety benchmark for Chinese large language models.
SuperGLUE
by SuperGLUE team (NYU, Facebook AI, University of Washington and DeepMind)
SuperGLUE benchmark site: harder successor to GLUE with language-understanding tasks, toolkit and leaderboard.
SWE-bench
by SWE-bench team (Princeton University and Stanford University)
SWE-bench leaderboards: coding-agent results on resolving real GitHub issues, across SWE-bench Verified, Multilingual and Multimodal.
SWE-bench Pro
by Scale
Scale AI SWE-bench Pro GitHub repo: benchmark of long-horizon software engineering tasks for agents.