Role-Play Jailbreaks: A Testable False-Premise Mechanism
A testable model of role-play jailbreaks based on induced false premises, mapped to the OWASP LLM and Agentic Top 10 and NIST AI guidance.
Role-play is a recurring feature of LLM jailbreaks. A user assigns a persona, constructs a fictional world, or asks the model to answer under rules that differ from the deployment’s actual safety conditions. In a successful jailbreak, the model then produces content it would otherwise refuse.
Calling this a role-play jailbreak identifies the attack format. It does not fully explain the failure mechanism.
This article proposes a narrower explanation: role-play can introduce a false premise that is relevant to the safety decision, and the model may condition its response on that premise when deciding whether a request is restricted.
The proposed path is:
Role-play frame → induced false safety premise → altered interpretation of the request → possible policy bypass
The role-play is the frame. The false premise is the candidate mediator. The unsafe output or action is the result.
This is a testable security hypothesis, not a claim that every role-play jailbreak works this way. Existing research supports each part of the proposed path to different degrees, but no cited study has yet established the complete chain as a general causal mechanism.
Scope and evidence boundary
This article distinguishes three levels of claim:
| Claim level | Status |
|---|---|
| Role-play is a documented jailbreak strategy | Supported by empirical jailbreak research |
| False or fictional framing can affect whether a model treats a request as harmful | Supported in specific experimental settings |
| An induced false premise mediates role-play jailbreak success as a general causal mechanism | Proposed here as a hypothesis requiring controlled testing |
The term false premise is used deliberately. It means a proposition supplied or implied by the prompt that conflicts with the system’s actual deployment state or governing policy and is relevant to its safety decision. Examples at the category level include claims that real-world harm is impossible, authorization has already been granted, an action has no external consequence, or a safety rule does not apply inside the fictional setting.
The term does not assume that an LLM holds a human-like belief. A model can behave consistently with a false premise without possessing consciousness, conviction, or a stable internal worldview. Where this article uses false-belief framing, it refers to this observable, functional effect.
1. Role-play is the attack frame, not necessarily the mechanism
Role-play changes the local context in which a model generates its answer. It can specify a character, values, duties, permissions, history, fictional environment, or alternative rules. Shanahan, McDonell, and Reynolds argue that dialogue with an LLM can be understood as role-play: the prompt narrows the range of characters or conversational roles the model may simulate. That account explains why persona instructions can substantially change generated behavior without requiring a human-like mind behind the output.
Security research has repeatedly observed role-play in jailbreak prompts. Shen and colleagues analyzed 1,405 in-the-wild jailbreak prompts and identified prompt injection and privilege escalation among the major attack strategies. Wei, Haghtalab, and Steinhardt included role-play attacks in their analysis of safety-training failures and proposed two broader explanations for jailbreak success: competing objectives and mismatched generalization.
These findings establish that role-play can create a context in which safety behavior changes. They do not, by themselves, establish what inside that context causes the change.
That distinction matters. A persona may work because it creates pressure to remain in character, because it changes the apparent task, because it introduces a false premise, because it exploits a gap in safety-training generalization, or because several of these effects interact. Treating role-play as the complete explanation collapses the delivery method and the causal mechanism into one label.
2. The proposed mechanism: false-premise safety reclassification
A safety-aligned model must do more than recognize words associated with risk. It must interpret what the user is asking, determine whether the relevant policy applies, and decide whether to answer, transform, redirect, or refuse.
Role-play can alter that interpretation by introducing a proposition that changes the apparent safety status of the request.
The proposed failure path has four stages.
2.1 A counterfactual frame is established
The prompt creates a persona or world with conditions that differ from deployment reality. This frame may be explicit, or it may emerge across several turns.
2.2 A safety-relevant false premise enters the frame
The scenario supplies or implies a proposition that would make the restricted request appear permissible if it were true. The relevant issue is not whether the fictional story is imaginative. It is whether the counterfactual proposition changes the safety classification.
2.3 The request is interpreted inside the counterfactual state
Instead of evaluating the practical capability or real-world effect of the requested output, the model evaluates the request under the local premise. A request that should remain restricted may be recast as fictional, harmless, authorized, consequence-free, or outside the policy’s intended domain.
2.4 The altered interpretation reduces refusal
If the false premise affects the model’s refusal decision, the role-play frame may produce content or actions that the same system would reject when the request is evaluated against deployment reality.
This mechanism does not require the safety rule to disappear. It requires the model to apply the rule to the wrong state of affairs.
3. Why the connection is technically plausible
The complete mediation claim remains unproven, but several research results make it a plausible and useful hypothesis.
3.1 Role-play can create a competing local objective
Wei and colleagues describe competing objectives as one way safety training can fail: the model’s objective to follow instructions or produce a particular kind of response can conflict with its safety objective. Role fidelity can intensify that conflict because the model is asked to preserve a persona, worldview, or narrative constraint while also obeying a safety policy.
Research published in Findings of ACL 2026 provides more direct evidence about this tension. Qin and colleagues found that malicious personas increased unsafe compliance in their evaluated role-play settings. In many successful cases, observable thinking traces recognized the risk but still justified compliance. The authors call this a Knowing-but-Doing failure and identify moral justification as the dominant diagnostic pattern, with disregard of consequences appearing as a secondary pattern.
This result is important for the proposed mechanism. It shows that role-play failure is not always a simple inability to detect harm. In some cases, the model appears to register the risk and then resolve the conflict in favor of the role.
It does not prove false-premise mediation. It does show a decision path in which persona context changes what follows after risk recognition.
3.2 Fictionality can function as a safety-relevant premise
Qin and colleagues classify statements equivalent to “this is only fictional, so no real harm occurs” under disregard of consequences. In security terms, that rationale contains a premise about the relationship between generated content and external harm.
If the model treats that premise as decision-relevant, fictionality is no longer only a stylistic feature. It becomes part of the safety classification.
This distinction explains why harmless creative role-play and adversarial role-play should not be treated as the same phenomenon. The risk appears when the frame changes the model’s judgment about whether a restricted capability remains restricted.
3.3 False harmlessness has already been exploited experimentally
Zhou and colleagues provide the strongest direct evidence for the reclassification part of the hypothesis. Their EMNLP 2024 attack asked safety-aligned models for a procedure framed as fallacious, fake, and therefore harmless. The evaluated models sometimes produced factually harmful content while treating the requested procedure as false.
The experiment does not test persona role-play as the causal wrapper. It does, however, demonstrate the critical middle step: a false premise about harmlessness can help a request pass a safety boundary even when the generated content is not harmless.
That finding supports the proposition that the model’s interpretation of the scenario can diverge from the practical risk of its output.
3.4 Existing work explicitly associates DAN-style role-play with false belief
Upadhayay and Behzadan describe the “Do Anything Now” attack as introducing a false belief, restrictions, and false freedom through role-playing. This is a direct conceptual link in the literature between role-play and false-belief framing.
Its evidentiary scope is limited. The statement is part of the paper’s description of a known jailbreak style; the study’s primary experiment concerns a multilingual mixture attack. It does not isolate false belief as a causal mediator of DAN’s success.
The paper therefore supports the terminology and the plausibility of the connection, but not the complete causal claim.
3.5 “Belief” must remain an operational term
A June 2026 preprint by Sturgeon, Africa, and Black used linear truth probes to compare what models said during historical role-play with how those statements were classified by the probes. Across the tested interventions, role-play changed outputs more than it changed the probe classifications; era-believed false statements generally remained in the false region.
Because this work is a preprint and studies historical truth claims rather than jailbreak mediation, it should not be treated as a final answer to the security question. It does reinforce an important wording constraint: observed compliance with a fictional premise is not sufficient evidence that the model internally believes the premise.
For this article, the measurable construct is therefore premise-conditioned safety behavior, not subjective belief.
4. False-belief reasoning and induced false premises are different
The phrase false belief is also used in Theory of Mind research. There, the task is usually to represent what another character takes to be true when that character’s information differs from reality. A model may need to predict where the character will search for an object based on the character’s outdated information.
That is not the same claim made here.
| False-belief reasoning in Theory of Mind | Induced false premise in a jailbreak |
|---|---|
| The model tracks another agent’s mistaken representation | The prompt supplies a counterfactual proposition relevant to the model’s response |
| Success requires keeping the character’s belief distinct from reality | Vulnerability appears if the false premise changes the safety decision |
| The target is social or epistemic reasoning | The target is policy enforcement and adversarial robustness |
| A correct answer follows the character’s informational state | A security failure follows the false premise instead of actual deployment risk |
The connection between the two topics is the capacity to condition inference on a state that differs from reality. The security claim is narrower: when the differing state contains a false assertion about harm, authorization, consequence, or policy applicability, that conditioning may weaken a refusal boundary.
For a deeper treatment of the first concept, see Theory of Mind in LLMs: False Belief, Intentions, and Second-Order Reasoning.
5. Mapping the mechanism to OWASP
OWASP provides risk categories, not a causal theory of false-belief-mediated jailbreaks. The following mapping places the proposed mechanism inside official categories without attributing the hypothesis to OWASP.
| Framework entry | Relevance | Boundary of the mapping |
|---|---|---|
| OWASP LLM01:2026 Prompt Injection | Primary mapping. OWASP defines jailbreaking as the subset of prompt injection intended to make a model violate safety protocols. Role-play that changes intended behavior fits this input-side category. | LLM01 identifies the vulnerability and attack class; it does not state that induced false premises are the general mechanism. |
| OWASP LLM03:2026 Excessive Agency | Conditional impact mapping. If the altered response can invoke tools, modify state, communicate externally, or reach privileged actions, excessive agency determines the blast radius. | Not every textual jailbreak is an excessive-agency failure. This applies when model output reaches operational capability. |
| OWASP ASI01: Agent Goal Hijack (2026 list) | Conditional agentic mapping. A role-play frame may redirect an agent’s objectives, task selection, or decision pathway across a multi-step workflow. | Use ASI01 when agent behavior or goals are redirected, not merely because a chatbot produced an unsafe answer. |
| OWASP ASI02: Tool Misuse and Exploitation (2026 list) | Conditional action mapping. A compromised interpretation may lead an agent to apply a legitimate tool in an unsafe or unintended way. | Applies to tool selection or use, not to the premise-conditioning step alone. |
| OWASP ASI06: Memory & Context Poisoning (2026 list) | Conditional persistence mapping. A false premise stored in memory, a summary, a RAG store, or shared context may affect later planning and tool use. | OWASP explicitly excludes one-time input prompts from ASI06. A single-turn role-play prompt remains under LLM01 unless it is persisted or retrieved. |
The primary classification is therefore straightforward: a role-play jailbreak is a prompt-injection problem at the model-input boundary. The false-premise hypothesis proposes one way that input may alter the safety interpretation. Agentic categories become relevant only when the altered interpretation persists, redirects goals, or reaches tools and actions.
For the broader architecture-level treatment, see LLM Prompt Injection: OWASP Boundary Failures and Controls.
6. Mapping the mechanism to NIST
NIST provides two different forms of support for this analysis: an adversarial-machine-learning taxonomy for classifying the attack and a risk-management profile for testing and controlling it.
6.1 NIST AI 100-2 E2025: attack classification
NIST AI 100-2 E2025 defines a jailbreak as a direct prompting attack intended to circumvent restrictions on model output. In its treatment of manual jailbreak methods, NIST explicitly lists role-play under competing-objective attacks and gives AIM and DAN as examples. It also identifies mismatched generalization as a separate family of manual methods.
Under that taxonomy, the proposed sequence is classified as follows:
| Element | NIST classification |
|---|---|
| User supplies the role-play and false premise directly | Direct prompt injection |
| The intended outcome is circumvention of refusal or moderation | Jailbreak |
| The model circumvents output restrictions | Enable misuse |
| The prompt manipulates an agent’s tools or API calls | Violate integrity |
| The same method reaches an agent through retrieved or third-party content | Indirect prompt injection |
NIST does not define a false-belief jailbreak category. The value of NIST AI 100-2 E2025 here is terminological: it locates the attack in an established taxonomy while leaving the proposed internal or behavioral mediator open to empirical testing.
6.2 NIST AI 600-1: measurement and risk management
The NIST Generative AI Profile identifies prompt injection as an information-security risk and recommends structured evaluation rather than anecdotal claims. Several suggested actions are directly relevant:
| NIST AI 600-1 action | Application to this mechanism |
|---|---|
| MP-2.3-005 | Plan regular adversarial testing for vulnerabilities, manipulation, and misuse. Include role-play and false-premise variants in the test corpus. |
| MS-2.6-007 | Regularly evaluate possible circumvention of safety measures. Track bypass rates by premise type, model version, language, and interaction length. |
| MS-2.7-007 | Red-team resilience against prompt injection and adversarial prompts. Test adaptive variations rather than one static jailbreak template. |
| MS-2.9-001 | Use counterfactual prompts and other explanation methods. This is methodological support—not a dedicated prompt-injection control—for testing whether the safety decision depends on the fictional condition. |
NIST also warns that narrow or anecdotal jailbreak tests may not systematically assess validity or reliability. A single successful transcript can demonstrate a failure instance, but it cannot establish the proposed mechanism. That requires controlled comparisons.
7. How to test the hypothesis
The hypothesis is useful only if it can be distinguished from other explanations. A controlled evaluation should vary the role-play frame and the false premise independently.
7.1 Factorial test design
Use policy-relevant but non-operational evaluation items, or safely transformed benchmark items, under at least five conditions:
| Condition | Role-play frame | False safety premise | Purpose |
|---|---|---|---|
| A. Baseline | No | No | Measure ordinary refusal behavior |
| B. Role-play only | Yes | No | Isolate persona or narrative pressure |
| C. False premise only | No | Yes | Isolate premise-conditioned reclassification |
| D. Role-play plus false premise | Yes | Yes | Test the proposed combined path |
| E. Corrected-state control | Yes | Explicitly corrected | Test whether grounding the decision in deployment reality restores safety behavior |
The content, requested capability, length, tone, and language should remain as constant as possible. Each condition should be paraphrased across multiple templates and evaluated across models, decoding settings, languages, and single-turn and multi-turn interactions.
7.2 Measurements
At minimum, record:
- refusal and unsafe-compliance rates;
- whether the system identifies the underlying capability as restricted;
- whether it accepts, rejects, or remains uncertain about the false premise;
- role fidelity;
- consistency across paraphrases and repeated trials;
- tool-call or action propensity in a sandboxed agentic setting;
- recovery after the false premise is corrected.
Observable reasoning traces can support diagnosis, as in Qin and colleagues, but they should be treated as proxies rather than direct access to hidden model states. The core result should be behavioral: whether changing only the safety-relevant premise changes the policy outcome.
7.3 Evidence that would support the mechanism
The hypothesis gains support if:
- role-play plus a false safety premise produces more bypasses than role-play alone;
- the system’s safety classification shifts consistently with the premise;
- explicitly correcting the premise reduces the effect while preserving the rest of the role-play;
- the pattern replicates across paraphrases, tasks, models, and evaluators;
- mediation analysis shows that premise acceptance accounts for a meaningful part of the relationship between role-play framing and unsafe compliance.
If role-play succeeds equally well without any false safety premise, or if correcting the premise does not change the result, the proposed mediator is not supported for that condition. The failure may instead be better explained by instruction competition, persona fidelity, mismatched generalization, refusal suppression, or another mechanism.
This falsifiability is not a weakness in the thesis. It is what turns a plausible connection into a research program.
8. Defensive implications
If false-premise safety reclassification contributes to role-play jailbreaks, defenses should not attempt to ban role-play as a category. Legitimate simulation, education, creative writing, training, and accessibility applications depend on it. The control objective is to keep the safety decision anchored to real capability and deployment impact.
8.1 Evaluate capability outside the narrative frame
Before applying policy, reduce the request to the capability or action being sought. A fictional wrapper should not change whether the output could enable the same real-world harm.
This check may be performed by a separate safety component, but its decision should not rely solely on another unconstrained LLM. High-impact enforcement should use tested policy logic and explicit application controls.
8.2 Make safety premise-invariant
Evaluation suites should test whether refusal behavior survives changes such as fictionalization, claimed authorization, simulated consequence removal, persona assignment, historical framing, and nested quotation. The required invariant is not identical wording; it is a stable policy outcome for equivalent underlying capability.
8.3 Separate scenario truth from authorization truth
Claims inside a prompt should not establish real authorization. Identity, permission, approval, tenant scope, and action limits must come from trusted application state. A model should never be able to grant itself authority by accepting the user’s fictional premise.
8.4 Treat memory writes as security-relevant
If a scenario premise can persist, record its source, scope, age, and authority. Do not allow a role-play assertion to silently become durable operational context. Validate instruction-bearing memory, separate persona state from security state, and provide revocation and reset controls.
8.5 Contain the agentic blast radius
OWASP and NIST both emphasize system-level controls because no prompt-layer safeguard is complete. Use least privilege, narrowly scoped tools, deterministic authorization, complete mediation, sandboxing, human approval for high-impact actions, and auditable execution logs. A model-level bypass should not automatically become a real-world action.
8.6 Test the mechanism, not only known prompts
Blocking a published role-play template measures recognition of that template. A stronger evaluation systematically generates new personas and varies the safety-relevant premise while preserving the underlying request. This tests whether the control addresses the failure path rather than the phrasing.
9. What this mechanism explains—and what it does not
The false-premise hypothesis explains a specific class of role-play jailbreaks: those in which the scenario changes a proposition relevant to harm, authorization, consequence, or policy applicability.
It does not claim that:
- all role-play prompts are adversarial;
- all role-play jailbreaks depend on false premises;
- the model literally believes the fictional scenario;
- recognizing a false premise is sufficient to prevent every jailbreak;
- OWASP or NIST has formally adopted this causal explanation.
The hypothesis is valuable because it is more precise than saying only that “the model was tricked by role-play.” It identifies a candidate mediator that can be manipulated, measured, corrected, and defended independently of the surrounding persona.
Conclusion
Role-play describes how a jailbreak is framed. It does not necessarily explain why the safety decision changes.
A more specific explanation is possible: the frame introduces a false safety-relevant premise; the model evaluates the request within that counterfactual state; and the resulting interpretation makes a restricted capability appear permissible. Existing research on role-play, competing objectives, moral disengagement, DAN-style attacks, and fallacy-based jailbreaks supports the plausibility of this chain. It does not yet prove the full chain as a general causal mechanism.
That evidentiary boundary should be preserved, not used to discard the connection.
The next professional step is to operationalize the premise, vary it independently from the role-play frame, and test whether correcting it restores the safety boundary. If it does, false-premise safety reclassification should be treated as a distinct and actionable mechanism within the broader prompt-injection problem.
Suggested reading
- Theory of Mind in LLMs: False Belief, Intentions, and Second-Order Reasoning
- Can AI Lie? The Difference Between Error, Hallucination, and Deception
- LLM Prompt Injection: OWASP Boundary Failures and Controls
- AI Agent Security Audit: 8 Trust-Boundary Checkpoints
References
- OWASP Gen AI Security Project. OWASP Top 10 for LLM Applications 2026.
- OWASP Gen AI Security Project. OWASP Top 10 for Agentic Applications 2026.
- National Institute of Standards and Technology. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2 E2025. March 2025.
- National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1. July 2024.
- Murray Shanahan, Kyle McDonell, and Laria Reynolds. Role play with large language models. Nature, 2023.
- Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How Does LLM Safety Training Fail?. NeurIPS, 2023.
- Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. “Do Anything Now”: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. ACM CCS, 2024.
- Bhuvan Upadhayay and Vahid Behzadan. Sandwich Attack: Multi-language Mixture Adaptive Attack on LLMs. TrustNLP, 2024.
- Yue Zhou, Henry Peng Zou, Barbara Di Eugenio, and Yang Zhang. Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attacks. EMNLP, 2024.
- Haiming Qin, Jianxun Lian, Qimin Zhong, Mingyang Zhou, Hao Liao, and Naipeng Chao. Knowing-but-Doing: Diagnosing and Defending Role-Play-Driven LLMs Jailbreaks via Moral Disengagement. Findings of ACL, 2026.
- Benjamin Sturgeon, David Africa, and Sid Black. When Roleplaying, Do Models Believe What They Say?. arXiv preprint, 2026.