Plot cosine similarity of each component vs tok0, per token. One panel per component (excluding 'sum'). Shows cos(comp[tok0], comp[tok_i]).

Activation-only energy fractions and frac(gate>0) at W_down top-k. Three panels per output dim (all use mean-of-per-sample): Left: up_out

Bar plot: each residual stream component at given dims, stacked vertically. pack: dict from _collect_residual_stream_components.

Per output dim: ungated vs gated projection onto W[dim,:]. Left panel: original (causal) attention. Right panel: self-attend-only (ablated) a

1-row grid per output dim: E[||down_in||], cos(E[down_in], W), E[dot(down_in, W)]. Uses per-sample norm stats for ||down_in|| (mean-of-per-sample

Bar plot showing how down_in outlier dims are created from gate_out * act_out * up_out. For each down_in outlier dim, shows 4 grouped bars per to

Bar plot: rms_in and rms_out values at specific dims across all tokens. Multiple dims are arranged vertically (one row per dim).

Bar plot: per-token E[||rms_in||] with std error bars on tok0. Uses per-sample norm stats (mean-of-per-sample norms).

Plot W_down row distribution for each output dim. Layout: one row per output dim, single panel: sorted |W_down[odim, :]| with cumulative en

12=0,42=1' -> [(12,0),(42,1)]

Collect mean embedding and post-attention residual (embed + attn_out) per token.

Find dims where |tok0| < ratio * mean(|tok1..T|).