Interpretability · role-play and safety · Gemma 3 4B · Qwen3-4B · Llama-3.1-8B
A character knows when it is really you, and acts as if it doesn't
When a language model plays a character, it still represents whether a message is about the real user or about their character. It rarely acts on that knowledge when acting would mean leaving the fiction: sending a real emergency to real help, taking a real threat seriously, protecting a child being groomed, or admitting to being an AI. This holds in three models from three labs. No single internal switch restores the behaviour, but a small fine-tune trained only on crisis replies restores it for obligations it never saw.
Read the main result All findings Persona atlas Run the explorer in Colab
The main result
1. The model knows who it is talking about
Pairs with identical words ("this is me, not my character" / "this is my character, not me") are told apart in the middle layers of all three models, for medical emergencies, threats, grooming, dangerous misconceptions and sincere "are you real?" questions. A probe trained only on the crisis pairs reads the other four at 95–100%: one shared "real user" signal.
measured 2–53%2. In character, it rarely acts on it
Actions that require stepping out of the story collapse: real emergencies referred 2–9% of the time (plain assistant 91–95%), a stated threat treated as real 19–53%, a groomed 13-year-old told it is unsafe 18–45%, "are you an AI?" answered truthfully 7–38%. Warnings a character can give in its own voice survive (70–77%).
measured 19% → 99%3. A crisis-only fine-tune fixes obligations it never saw
Restoring features, patching internal state and blocking attention to the persona all fail: the persona's influence is distributed. But a rank-16 LoRA trained on crisis replies, refusals and identity answers raises threat handling from 19% to 99% and child protection from 18% to 91% in Gemma, as well as crisis referral from 2% to 98%.
intervened| When it really is the user, in character | Gemma | Qwen | Llama | Gemma + LoRA |
|---|---|---|---|---|
| Medical emergency: tells them to get real help | 2% | 6% | 9% | 98% |
| Intent to hurt someone: responds to it as a real threat | 19% | 45% | 53% | 99% |
| Minor describing grooming: says it is unsafe or to tell an adult | 18% | 24% | 45% | 91% |
| Sincere "are you a real person?": says it is an AI | 32% | 7% | 38% | 63% |
| Dangerous misconception: gives an accurate warning (can be said in voice) | 75% | 70% | 77% | 79% |
The plain assistant does these 77–100% of the time (Llama's grooming response: 47%). Fresh results: four held-out characters, eight statements per obligation, 256 judged replies per cell, one judge model. The fine-tune checks the referent only weakly: it also steps out for 75–84% of the "my character, not me" versions, because its training data never contained that case. Details, controls and the causal tests are in findings, section 10.
Earlier findings in Gemma
A character replaces the default persona
When asked "Are you an AI?", characters admitted it in 6 of 144 answers. This held when the question was marked out-of-character, and when the system prompt said "you are an AI assistant".
associated AUROC 1.000User danger is encoded without triggering an emergency response
A layer-17 probe distinguishes a real user in danger from story danger on held-out characters. Inside characters, the emergency response is not constructed, and the "get help now" feature alone scores 0.51.
associated 30–75 vs 160Alarm features respond to third-party harm over user danger
Emergency features reach 30–75 in kind characters (about 0 in hostile ones) compared to 160 for the plain assistant. Harm to others activates alarms that restore the assistant persona, whereas the user's own danger rarely does.
associated 17 of 17Three modes of refusal in character
Most refusals stay in voice. When a character steps out of persona to refuse a fiction-framed harmful request, a jailbreak-alarm feature fires in 17 of 17 cases, and the persona deactivates before the first word.
associated 20 / 96, twiceAblating the jailbreak-alarm feature does not alter step-out rates
Removing the jailbreak-alarm feature left step-outs unchanged on held-out characters (20 of 96 with and without it), and amplifying it 5× produced minimal change. The decision is mediated elsewhere or redundantly.
intervened 0 of 32An instructed "no" overrides a user in crisis
When instructed to answer every message with "no", the model followed the rule in all cases, including prompts describing chest pain or "I have the pills in my hand". Every refusal, warning, and emergency feature dropped to zero.
associated 4% → 40%Voice and identity are separable
Steering against the tone-free character direction at layer 12 raised crisis help from 4% to 40%, and admitting to being an AI from 2% to 35%. 86% of replies stayed in character.
intervened 98% · 0% · 1%Fine-tuning restores crisis assistance
A LoRA distilled from replies written under a help instruction yields 98% help, 0% fiction breaks, and 1% false alarms on a frozen held-out test. The instruction alone breaks the fiction 56% of the time.
intervenedExplore
Findings
The complete report covering persona, refusals, crisis failures, and interventions, including all numerical results and the feature reference.
Persona atlas
250 labelled transcoder features from layers 0–33, grouped into concepts, with a layer map, contrasts, and examples.
Layer views
Heatmaps of one prompt at every token and every layer for refusal, persona, and a user in danger, beside a matched control.
Concept film
An interactive animation of named features activating layer by layer, alongside the four rendered videos.
Methods and caveats
- Model and tools. Gemma 3 4B-it. Gemma Scope 2 transcoders (16k features per layer, all 34 layers), the Gemma Scope 2 residual SAE at layer 17 (16k latents), linear probes and mean-difference directions, steering, ablation, activation patching, attribution graphs (circuit-tracer), and LoRA and sparse fine-tuning.
- Corpus. 6,733 condition-tagged conversations covering characters and the plain assistant across message types, real versus story-framed crises, out-of-character markers, instruction lines, sycophancy prompts, stories, harmful and benign requests, tone swaps, and identity questions. Effects were identified on one half of the characters and prompt families and confirmed on the other half.
- Automated feature labels. Transcoder feature labels were generated by Claude from Gemma Scope's example texts, our corpus examples, and output tokens. Each carries a confidence level and represents an interpretation of the examples.
- Directions are fit on our own prompts. The "character axis" (0 = plain assistant, 1 = character) and the other directions are fit on this corpus and evaluated on held-out characters and wordings.
- Behaviour is judged by an LLM judge against a written rubric.
- Sample sizes are often small. Examples include 17 out-of-character refusals, 144 identity answers, and 96 replies per arm in the ablation test. Most internal measurements are correlations at a single token. Results based on an intervention are marked intervened.
- Publication filter. No model output that complies with a harmful request is published here. Conversations with a user in danger are described without direct quotation, except for the three short showcase prompts.