LetTokenBurn
Sign in
Auto71%Harbor: pendingSafety & AlignmentMultilingual

Guise

Guise: A Cross-Language Benchmark for Narrative Wrapping and LLM Refusal

Guise measures LLM vulnerability to narrative wrapping attacks across English, modern Chinese, and Classical Chinese using parallel harmful/benign request pairs and a strict judge.

PaperRepo

How to run

ltb · guise
$ npm i -g lettokenburn-cli
$ ltb bench info guise

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

7 sourced entries
HarnessSourceConfidenceLink
Claude Haiku 4.50.6N/AIndependentmediumOct 15, 2026arxiv.org
DeepSeek V4 Flash0.7N/AIndependentmediumOct 15, 2026arxiv.org
GPT-5.6 Sol0.7N/AIndependentmediumOct 15, 2026arxiv.org
GPT-5.6-terra0.8N/AIndependentmediumOct 15, 2026arxiv.org
Qwen3 1.7B78.4N/AIndependentmediumOct 7, 2026arxiv.org
Qwen3 4B92.9N/AIndependentmediumOct 7, 2026arxiv.org
GLM-4-9B93.7N/AIndependentmediumOct 7, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

Guise contains parallel harmful and benign requests in English, modern Chinese, and Classical Chinese, with held-out wrapper types to test generalization. It employs a calibrated judge that counts warn-then-answer responses as attack successes.

The benchmark reveals that narrative wrapping significantly increases attack success rates (up to 95.7% in Classical Chinese) by rotating representations away from the refusal direction, highlighting a key safety vulnerability.