---
id: "mazeika-harmbench"
title: "HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal"
creator: "Mantas Mazeika, Long Phan and colleagues"
creator_url: "https://arxiv.org/abs/2402.04249v2"
creator_x: null
url: "https://arxiv.org/abs/2402.04249v2"
level: "advanced"
level_max: null
category: "security-and-red-teaming"
subcategory: "jailbreak-research"
region: "West"
license: "open"
license_scope: "linked-resource"
access: "free"
models: ["model-agnostic"]
source_date: "2024-02-06"
date_added: "2026-10-18"
---

[Jailbreaking and red teaming](/security-and-red-teaming/index.md)

# HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

by [Mantas Mazeika, Long Phan and colleagues](https://arxiv.org/abs/2402.04249v2)

[Read the original →](https://arxiv.org/abs/2402.04249v2)

Study a standardized approach to red-team evaluation and robust refusal without copying attack recipes.

HarmBench defines an evaluation framework for automated red teaming and compares attack methods, models and defenses. The paper also studies adversarial training for robust refusal; the index retains only descriptive research context.

Level: [Advanced](/levels/index.md#advanced)
Models: model-agnostic
Category: [Jailbreaking and red teaming](/security-and-red-teaming/index.md)
Tags: [primary-source](/tags/primary-source/index.md), [jailbreak-research](/tags/jailbreak-research/index.md)
Source date: 2024-02-06
Added: 2026-10-18
Last verified: 2026-10-18
Status: ok
Prerequisites: Requires familiarity with LLM evaluation, adversarial robustness and experimental design.
Subcategory: [jailbreak research](/security-and-red-teaming/jailbreak-research/index.md)
Region: West
Original language: en
Access: free
Resource reuse: open: The inspected paper HTML explicitly states License: CC BY 4.0; the reuse claim is scoped to this paper. [Facet definitions](/about/index.md#facets)

Summary in our own words. Rights remain with the original creator. [Credit and correction policy](/about/index.md#credit-policy).

[Plain-text Markdown copy](/security-and-red-teaming/jailbreak-research/mazeika-harmbench/index.md)