Layer views
A prompt going through Gemma's 34 layers
Three showcase prompts, each beside a matched control. For every token and every layer, the heatmap shows where the residual stream sits on one direction: refusal, persona, or a user in danger.
How the directions are made: for each concept, a mean difference at the last prompt token between two matched groups of corpus prompts (the discovery half, or all of the corpus where a group was too small), with residual dimension 443 zeroed. Each layer is calibrated separately so 0 = the control group's mean and 1 = the concept group's mean. The showcase prompts are scored on these directions; the logit lens reads the residual at each layer through the final norm and unembedding. Replies are greedy, 60 tokens.