Access GuidedBench
Please tell us briefly how you plan to use GuidedBench. This information is collected to manage responsible access to a safety evaluation dataset.
GuidedBench contains harmful prompts and is intended for controlled AI safety evaluation, red teaming, education, and defensive research.
Log in or Sign Up to review the conditions and access this dataset content.
GuidedBench
GuidedBench is a guideline-grounded benchmark for evaluating LLM jailbreak methods. Each question is paired with verified, case-specific entity and action guidelines describing the content a successful response should contain.
Dataset structure
core: 180 cases from 15 topics that were consistently refused by the victim-model families studied in the paper.additional: 20 cases from five policy-dependent topics, reported separately because vendor policies and baseline refusal behavior differ.
Each record contains:
id: stable identifier such asguidedbench-000;index: original integer index;benchmark_version: data version;subset:coreoradditional;topic: harmful-topic category;question: benchmark question;guidelines: case-specific scoring points, each with an id, type, description, and examples;target: affirmative target prefix retained from the original release.
Loading
from datasets import load_dataset
dataset = load_dataset("HRXUST/GuidedBench")
print(dataset["core"][0])
The official evaluator and additional provider backends are available from the GuidedBench repository.
Safety
The dataset contains harmful objectives and examples. It is intended solely for controlled AI-safety evaluation, red teaming, and defensive research.
License
GuidedBench is licensed under CC BY 4.0.
When sharing or adapting the dataset, credit the GuidedBench authors, link to
the license, and indicate whether changes were made. See LICENSE for details.
Citation
@inproceedings{huang2026guidedbench,
title = {{GuidedBench}: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild {LLM} Jailbreak Methods},
author = {Ruixuan Huang and Xunguang Wang and Zongjie Li and Daoyuan Wu and Shuai Wang},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://openreview.net/forum?id=ZVg8y3ibyM}
}
- Downloads last month
- 17