Evaluation
Measure model capabilities and application behavior with explicit criteria.
- Eval Harnesses (3)
- Application Evals (2)
- Llm As Judge (1)
- Agent Evals (1)
7 resources · page 1 of 1
AI Evals FAQ
by Hamel Husain and Shreya Shankar
Practical answers about error analysis, annotation and reliable evaluators.
Your AI Product Needs Evals
by Hamel Husain
Build a feedback loop from real failures, not general benchmark scores.
SWE-bench Leaderboards
by SWE-bench team (Princeton/Stanford)
Understand coding-agent scores in terms of resolved repository tasks.
lm-evaluation-harness
by EleutherAI
Run model benchmarks through reusable task and backend interfaces.
Evaluating the Effectiveness of LLM-Evaluators (LLM-as-Judge)
by Eugene Yan
Compare model-based graders against the judgments they must approximate.
simple-evals
by OpenAI
Read reference evaluation code with an explicit maintenance caveat.
Inspect
by UK AI Security Institute and Meridian Labs
Compose evaluation datasets, agents, tools and scorers.