Evaluation for advanced learners

4 resources · page 1 of 1

• lm-evaluation-harness

  by EleutherAI

  Run model benchmarks through reusable task and backend interfaces.

  Advanced · West · open resource · en

• Evaluating the Effectiveness of LLM-Evaluators (LLM-as-Judge)

  by Eugene Yan

  Compare model-based graders against the judgments they must approximate.

  Advanced · West · closed resource · en

• simple-evals

  by OpenAI

  Read reference evaluation code with an explicit maintenance caveat.

  Advanced stale · West · open resource · en

• Inspect

  by UK AI Security Institute and Meridian Labs

  Compose evaluation datasets, agents, tools and scorers.

  Advanced · West · closed resource · en

Plain-text Markdown copy