From op-catalog to decompilation — a research design

DISASSEMBLY.md reads a transformer’s attention as a first-order instruction set: per-head QK/OV opcodes, named idioms, a coverage scorecard (“% of attention legible”), causal validation, and cross-model replication. This document scopes the next phase: turning that disassembly into an executable decompilation — reconstructing the computation, not just labeling components — and argues the obstacle is the same entanglement the forge tax measures.

This is a design doc (a target + a metric + milestones), not a results writeup. It is the flagship that ideas (i) cross-model breadth and (ii) multilingual invariance de-risk: you want the disassembler proven on ≥3 architectures before you trust a single decompilation-coverage number.

The decompilation taken literally — see the pylm track: a whole small LLM reimplemented as a small pure-Python program + flat-file knowledge store (no neural-net code or concepts), validated against the corpus and the model. It makes the decompilable fraction a running artifact (~50% of GPT-2 in 49 lines; 56% of Pythia-14m; shrinking with scale) — the disassembly→decompilation arc reaching its endpoint, and the forge-tax ceiling visible as the half it can’t reach.

The gap: disassembly ≠ decompilation

  disassembly (have) decompilation (want)
unit one head / one idiom in isolation the composition — which ops chain into which
coverage metric % of attention legible (named or weight-binding) % of the forward pass faithfully reconstructable
MLP first-order read→write token catalog MLP neurons as first-class ops (key–value memories)
validation mean-ablation damages the metric (necessary) recompile the extracted program; KL ≈ host (sufficient)
output a human-readable listing a runnable reduced model + a symbolic trace

The disassembly already took the first two rungs of composition: the induction macro read as OV_A ∘ QK_B (prev-token head feeds the induction head; weight-composed diagonal vs behavior ρ≈0.78, path-patch-gated), and the IOI chain duplicate → S-inhibition → name-mover read as a product of Q-composition scores. Decompilation is the general algorithm those two are instances of.

The load-bearing idea: reconstruction coverage

Replace “% legible” with a metric that has teeth — does the extracted op-graph, compiled back into a runnable model, reproduce the host’s next-token distribution?

reconstruction_coverage(budget B) =
    1 − KL( host ‖ recompiled[ ops selected under budget B ] ) / KL( host ‖ mean-ablated-everything )

First results are live — and humbling. The reconstruction-coverage idea is now run across six models for the induction circuit and (GPT-2) the IOI circuit: Executable decompilation (keep the circuit’s heads, ablate the rest, measure induction-NLL / IOI logit-diff recovery + the budget curve) and the attention-vs-MLP substrate split. The honest headline: no small head-set is a sufficient decompilation — induction recovers ≤30% from its named 8 heads (even under the gentler resample-ablation), needs ~all heads to fully reconstruct, and leans on attention and the MLP substrate roughly equally; even IOI’s celebrated 26-head circuit is not sufficient in isolation. The named circuits are necessary and the dominant drivers, but the behaviour is distributed across the near-whole network. So a faithful decompilation here is not a tiny op-graph — the budget curve, not a single small circuit, is the real object.

Four work-items

1. The composition DAG extractor

Generalize rung-2/3 into the standard algorithm: from the weights, score every directed edge OV_g → QK_h (head g’s write feeds head h’s query/key/value) as a composition bilinear in the operand basis; threshold + path-patch-validate to keep live edges. Output: the model’s circuit DAG (heads + MLP nodes, typed edges Q/K/V-composition). The induction macro and IOI chain become the first two validated sub-DAGs; the extractor produces them automatically and finds new ones.

2. The MLP gap

The catalog is attention-centric; MLPs carry real computation (greater-than is MLP-dominated — the OV probe sees only its attention-side shadow). Treat MLP neurons as ops: in-direction (gate/up read) and out-direction (down write) in the operand basis = a key→value memory; cluster into named MLP idioms (detokenization, entity attributes, successor arithmetic). Add them as nodes in the DAG (1) and as keepable ops in the recompile (the metric).

3. The operand-basis ceiling

First-order disassembly uses centroid / SAE operands that are imperfect; composition only reads cleanly in a basis where the ops compose cleanly. This is where decompilation meets the rest of the program: the two-basis forge’s U_C (composition subspace) is exactly “the basis in which circuits survive.” (The specific writer-output U_C construction was retracted — see DISASSEMBLY.md — but the target stands: find the basis that makes reconstruction_coverage high at low budget.) polygram models the operand-dictionary geometry; the decompiler consumes it.

4. Executable verification & emission

Wire (1)+(2) into the recompile metric (the load-bearing idea), and emit the DAG in a checkable form: n-orca can express the extracted circuit as a typed neural DAG; larql can hold the decompiled model as a queryable index (“the model IS the database”). A decompilation is accepted only if recompile-KL clears a threshold — no unverified symbolic story counts.

The unifying claim (why this is one program, not a side quest)

Composition is not cleanly feature-readable for the same reason cov95 collapses under forging: the composition is entangled (the entanglement tower’s irreducible high-χ core).

So “how much can we decompile” and “how big is the un-forgeable core” are the same quantity measured from two directions — reconstruction_coverage from the disassembly side, 1 − cov95-tax / the tower’s convergent core from the forge side. The falsifiable prediction:

The reconstruction-coverage curve plateaus at the same fraction the entanglement tower identifies as the irreducible entangled core (~the low-χ substrate), and no op-budget closes the gap — because the residual is the entangled composition, which by the tower’s no-go is not decompilable into first-order ops. Decompilation has a hard ceiling, and that ceiling is the forge tax.

If true, this is the result: a single number — measured independently as (a) the decompilable fraction of the forward pass and (b) the forgeable/low-χ fraction — that bounds how interpretable a transformer can be made without retraining. If the curve instead reaches ~1.0, the entanglement story is wrong and the model is fully decompilable (also a publishable result).

Flagship result on real LMs — composition doesn’t factor through the SAE features (sae_forge_tax.py). The sharp prediction of the unifying claim, tested directly: force the residual through the SAE feature basis (decode∘encode, the forge bottleneck) and compare the damage to the composition (induction-NLL) vs the readout (generic next-token NLL). The feature basis taxes composition far more than readout — GPT-2: the full forge raises induction-NLL +1357% but generic-NLL only +70% (composition taxed ~19×, and taxed more in all 12 layers individually); Gemma-2-2B: +542% vs +399% (net +143%, 5/8 layers — weaker, as its induction is already distributed and its Gemma-Scope SAEs reconstruct loosely). This is the cov95 forge tax measured from the disassembly/reconstruction side: SAE features survive as readouts, the computation over them does not factor through them — the two contributions (forge tax / disassembly) meeting on one host. (Honest scope: SAE reconstruction is lossy everywhere, so the relative composition-vs-readout tax is the signal, robust at 12/12 layers in GPT-2; this is the reconstruction/NLL route, complementary to the cov95/mAUC route, not the full NativeModel weight-projection ceiling test of M4. runs/disassembly/sae_forge_tax_summary.json.)

What form is the core? (not an SAE, not a dense slab — a program)

“SAE vs dense” is a false dichotomy for the entangled remainder. The forge tax says the core has no sparse feature (SAE) basis in which composition is monosemantic — that rules out the SAE form. But the χ-ladder showed composition is factorable in weight/attention-composition coordinates (induction = OV_A∘QK_B; IOI = a head chain), and the instruction-tensor result found the program is low-rank (~5–11 templates vs ~132 random) — in operand-pair space, not residual-direction space (which is exactly why the residual- direction search, joint-U_C, failed). So the core’s natural form is a program: a compact DAG of reused weight-space bilinear templates over now-explicit operands. Subtracting the clean low-χ substrate is then a change of coordinates, not a sparsification — it makes the operands explicit so the remainder reads as “bilinears with clean arguments,” but it does not shrink the dense magnitude (the tower’s ~24% irreducible variance floor; the retrain no-go). Consequence for this program: the lever that “simplifies” the core is the composition-DAG extractor in operand coordinates (work-item 1+2), not a better SAE — and the reconstruction-coverage curve should plateau at the entangled-core fraction precisely because first-order feature ops can’t express the composition, while template ops over operands can. (Open at scale: whether the template count stays small and whether the irreducible floor itself grows with capability — the tower convergence was measured on small hosts.)

Core-rank result — the core is heavy-tailed low-rank, and CPU-simplifiable without retraining (core_rank.py). Measuring the empirical rank of each layer’s residual update (its contribution to the residual) over 6 models: the spectrum is heavy-tailed — a tiny dominant head (participation ratio 5–33, ~1–2% of d) but a real tail, so the functional rank (energy-95 / accuracy-preserving) is ~⅓ of d (hundreds of directions: rank-95 250→398→476 across the GPT-2 ladder), and the dominant-direction count grows with scale (PR 5→13→24) — consistent with the entangled core growing with capability, and with the forge tax’s “no tiny basis.” So the core decompiles into a moderate-rank structure, not ~5 templates. But it is substantially CPU-simplifiable for free: a no-retrain low-rank projection of every layer’s update onto its top-r PCA subspace is ~lossless at 70% rank in GPT-2 (≈ +0.05 NLL at 40%), and Gemma-2 is lossless at ~20% (its update is the lowest functional rank — ~80% of write-dims droppable), while Llama/Qwen are the least compressible (need ~70%+; higher functional rank). A concrete efficiency lever — rank-r per-layer factorisation, no retraining — with the compressible fraction architecture-dependent and shrinking with scale. (Caveat: this is the per-layer write-subspace rank, not the full input→output map; data-dependent PCA. runs/disassembly/core_rank_summary.json.)

And the layers SHARE that subspace. The cross-layer sharing measure: the union of all layers’ top-64 write- subspaces has effective rank 278 (GPT-2) / 517 (large) / 658 (Gemma) / 570 (Llama) — far below the no-sharing bound (nL·64, capped at d): the whole model’s per-layer writes collectively span only ~⅓–⅖ of d, and layer pairs are ~2× more aligned than chance (overlap 0.05–0.16 vs 0.03–0.08 random). So the entangled core is a shared moderate-rank subspace, not nL distinct ones — the strongest decompile/simplify lever yet: one shared ~300-direction basis captures most layers’ composition (decompile its structure; or a single global low-rank projection serves every layer on CPU). Sharing is partial (overlap ≪ 1) and grows with scale — compact, but not a few templates.

…but a single global basis is not a free lunch (honest refinement). Forcing every layer through the one shared union-basis at low rank is much worse than per-layer PCA (GPT-2 at 10% rank: +4.48 global vs +0.47 per-layer ΔNLL; Gemma +8.44 vs +0.38), only catching up at ~40–70% rank (Gemma’s global basis is even slightly better than per-layer at 70%, −0.39 — its sharing is strongest). So the partial overlap does not let you collapse the core to one tiny global basis: the efficient simplification is per-layer low-rank (lossless at ~70% GPT-2 / ~20% Gemma), with a moderate-rank global basis a second-best. The core is a shared-but-partial moderate-rank subspace — compact, reused across layers, but with real per-layer structure: “not a few templates, not a dense slab,” made quantitative.

What IS the shared subspace? — decompile its structure (core_basis_decompile.py)

Knowing the core is a shared ~300-direction subspace (“how big”) invites the deeper question: is that basis interpretable (“what is it”)? Three tests on the union basis Ug (top-K, K = its effective rank), GPT-2 s/m/l:

test what it asks result (gpt2 / medium / large) reading
(A) token/logit does Ug lie in W_U’s top-K logit subspace? 0.33 / 0.39 / 0.39 vs chance 0.36 / 0.40 / 0.40 at/below chance — the core is not the readout subspace; it is upstream compute
(B) operators is Ug built from the catalog ops’ OV-write subspaces? induction/prev-tok/dup/sink 0.39–0.47, random heads 0.39–0.46, chance 0.36–0.40 named ≈ random — the core is the aggregate of all writers, not a few named idioms
(C) logit-lens do Ug’s directions decode to coherent tokens? peak-z 5.2 / 5.9 / 5.1 vs random 4.6 / 4.8 / 4.5 readable — top dirs = function-words, punctuation, a verb axis, a register axis

So the entangled core’s shared basis is logit-lens-readable per direction, but is neither the readout subspace nor the named-operator basis — it is a broad write-basis whose individual directions decode to grammatical / lexical classes while collectively it is the sum of every layer’s writers. (runs/disassembly/core_basis_summary.json.)

Is the core a generic grammar? — the compact head is, the bulk is content (core_grammar.py)

The logit-lens (C) hint — directions decode to grammatical classes, not content — is the Chomsky hypothesis: is the core a content-free grammatical scaffold? A grammar core would be both generic (corpus-invariant) and grammatical (closed-class). Fitting Ug on three structurally distinct corpora (Shakespeare drama / a modern novel / Python source) and binning the shared directions by sharedness rank (GPT-2):

sharedness rank cross-corpus overlap vs chance closed-class fraction vs random verdict
top-16 (most shared) 0.44 vs 0.02 (22× chance) 0.28 vs 0.00 (28× base) generic AND grammatical — the grammar head
mid 16–64 0.23 vs 0.06 (4×) 0.03 fading
deep 64–K 0.35 vs 0.28 (~1×) 0.00 content (corpus-specific)
content tail 0.43 vs 0.36 (~1×) 0.00 content / rare-token (inventoryQuantity, _Lear)

The dissociation is clean: the top ~16 most-shared directions are both corpus-invariant (22× chance) and closed-class/grammatical (28× the base rate); everything deeper is neither. So there is a generic grammar — but it is a compact head of ~5–16 directions, not the whole Θ(d) core. This is the “simpler-than-Chomsky” version: a finite, content-free categorial scaffold (determiner-slot, punctuation-slot, verb-slot, pronoun-slot) — a learned distributional POS basis, not the recursive/hierarchical syntax of generative UG (which, if present, lives in the composition of these categories across positions — i.e. in the entangled bulk that pays the forge tax, not in the static write-basis). And it is learned, not innate — a generic transformer grows the scaffold from data with no syntactic prior. Cross-architecture (RoPE): Llama-3.2-1B reproduces the dissociation — top-16 directions 0.41 vs 0.01 chance (41×) corpus-invariant and 0.30 vs 0.01 (30×) closed-class (with a clean reflexive-pronoun direction _himself _ourselves _herself _themselves and a punctuation direction visible through the multilingual-byte noise), everything deeper neither. The grammar head is architecture-general, not a GPT-2 quirk. (runs/disassembly/core_grammar_summary.json.)

Big-O of the core: Θ(model size), not a few templates

Putting the rank results together answers “how many templates / what scaling”: the functional per-layer rank is Θ(d) (rank-95 ≈ ⅓–⅖·d at every scale: 250/768, 398/1024, 476/1280, 509/2304 — a constant fraction, growing with width), the shared union basis is Θ(d) (K ≈ ⅓–⅖·d), so the full per-layer composition is ~Θ(nL·d) directions ≈ Θ(nL·d²) parameters — a constant fraction of the model (the forge-tax floor). The “few templates” (the dominant-variance head, PR 5–33) is only the tip and grows slowly (~O(nL)); the grammar head is smaller still (~O(1), ~16 directions). Net: low-rank truncation buys a constant factor (~⅓–⅔), not a big-O reduction; and the irreducible core scales with the model, consistent with “the entangled core grows with capability.” The pylm sister track makes the consequence runnable — a flat-file grammar idiom decompiles the compact scaffold (see pylm track), but adds little to the token-level decompilable fraction (grammar is categorial; the n-gram modes already absorb it), and the un-decompiled ~50% is exactly the content that is neither n-gram nor relational fact — the entangled composition, the forge tax restated.

The core as a TENSOR NETWORK — MPS bond-dimension levels + the composition graph (core_mps.py)

The per-layer write rank is Θ(d) (above), but that is the single-cut rank. Treating the sequence of per-layer updates (Δ₁ … Δ_nL) as a state over layer-“sites” and measuring the bond dimension χ across each layer cut (the effective rank of the cross-correlation between layers ≤ L and > L — the q-orca / entanglement-tower view) gives a very different number, and the cleanest CPU-simplification result yet:

GPT-2 layers mean χ χ range across cuts max-bond mid-stack χ / max-bond (mid)
gpt2 12 15.9 15–16 144 0.11
gpt2-medium 24 15.5 14–16 288 0.056
gpt2-large 36 16.4 14–18 432 0.037

The bond dimension’s participation ratio is ~16 and FLAT — independent of depth and width. Adding layers to either side of a cut does not raise the dominant cross-cut coupling: an area-law signature on the coupling spectrum. Honest correction (see the TT surrogate below): χ≈16 here is the participation ratio of the cross-cut correlation spectrum — a coupling-concentration measure dominated by a few high-variance directions — not the rank you need to reconstruct the computation (exactly the core_rank gap: participation ratio ≈ 16, but rank-95 ≈ 250). So “the layers communicate through a 16-dim channel” is true of the dominant coupling, false as a runnable state dimension. The honest statement: the cross-layer coupling is heavy-tailed-low-rank (dominant head O(1), flat with depth), the runnable bond is Θ(d) (next), and the simplification lever stays core_rank’s per-layer low-rank.

…and the TT surrogate confirms it — the runnable bond is Θ(d), not 16 (core_tt.py)

Turning the bond into a runnable, no-retrain surrogate settles the CPU question. The residual carries two things — the token EMBEDDING (full-rank, injected once) and the inter-layer COMPUTATION — so the faithful tensor-train protects resid_0 in full and forces only the carried computation through a χ-dim bond: resid_{L+1} ← resid_0 + B_L Bᵀ_L (resid_{L+1} − resid_0). ΔNLL vs the full model (generic next-token):

ΔNLL χ=16 χ=64 χ=256 reading
TT (emb-protected bond) gpt2 / med / large +1.37 / +2.94 / +3.11 +0.77 / +1.94 / +2.41 +0.23 / +0.75 / +0.98 the χ≈16 bond is unusable; runnable bond ~⅓·d
per-layer (core_rank) +1.15 / +2.09 / +2.71 +0.54 / +1.14 / +1.34 +0.10 / +0.27 / +0.36 better at every χ — the real lever
resid (full-residual control) +1.44 / +2.90 / +3.14 ≈ TT ≈ TT emb-protection barely helps

Three honest findings: (1) at the area-law bond χ≈16, NLL is badly degraded (+1.4 to +3.1) — the participation ratio is not a runnable state size; (2) the runnable no-retrain bond is ~⅓·d, and per-layer truncation beats the running-bond TT at every χ (the TT compounds error across cuts, so it gets worse with depth: χ=256 ΔNLL +0.23→+0.75→+0.98 as layers go 12→24→36 — the opposite of “bigger = more compressible”); (3) protecting the embedding barely helps (TT ≈ resid control), so the un-compressible content is the computation, not the tokens. So the only free CPU lever is core_rank’s per-layer rank-⅓·d — a constant-factor (~3×) FLOP saving, lossless, no retrain — NOT a χ≈16 tensor-train collapse. The hard floor is the Θ(d) core (the forge tax): the irreducible composition is a constant fraction of the model, so no fixed-size surrogate beats the model’s own size without retraining. (A retrained TT — learn the cores instead of PCA-projecting activations — could reach a lower bond; that is the sae-forge feature-native direction, outside the no-retrain frame. runs/disassembly/core_tt_summary.json.)

…but with RETRAINING the floor falls ~30× — the core is not irreducible (core_distill.py)

The no-retrain results above all freeze the weights and use a linear, fixed (PCA) basis — so the Θ(d) floor they hit says nothing about whether a learned low-rank representation exists. This builds one: a rank-r bottleneck on each layer’s update, initialised from PCA (step 0 = the no-retrain floor) and then trained (base model frozen, only the bottleneck factors, ~300 steps). The floor collapses:

GPT-2 rank (per-layer update) no-retrain ΔNLL trained ΔNLL
8 (1% of d) +1.78 +0.03 (≈ lossless)
16 (2%) +1.28 −0.10
32 (4%) +0.97 −0.18
256 (33%, adaptation control) +0.24 −0.00

A trained rank-8 update (1% of d) is lossless, vs the no-retrain floor of ~⅓·d (256) — a ~30× rank reduction. The rank-256 control trains to ΔNLL ≈ 0 (no meaningful domain-adaptation bonus), so the comparison to the full model is fair: trained rank-8 genuinely matches it. So the “Θ(d) entangled core” was an artifact of freezing + linear PCA, not a property of the function — exactly the right caution: the no-retrain experiments measure the wrong thing for “is the core compressible.” Scale nuance: gpt2-large needs more rank/steps (rank-8 recovers 76%, rank-64 88% in 250 steps; rank-512 overfit — an optimization artifact of a huge bottleneck under-trained), so the lossless rank grows modestly with size and training budget — but it stays far below the no-retrain floor. Honest scope: this compresses the per-layer update (the residual write), not the full internal attention+MLP FLOPs, and uses a small eval (20 chunks) + cheap training; a rigorous version (more steps, general eval, factoring the output projection for real FLOP savings) is the natural follow-up. But the qualitative claim is settled: detangling/compressing the core is tractable with learning — “irreducible” is falsified; only the frozen-linear route was blocked. (runs/disassembly/core_distill_summary.json.)

The retrained-rank scaling law across the ladder (matched config: 300 steps, lr 1e-4)

Running the same rank-r retrained bottleneck across a 7-model ladder (Pythia 14m/70m/160m/410m + GPT-2 small/medium/large) at one config gives the scale-law of the lossless retrained rank — the rank where the trained ΔNLL first reaches base (≤ 0 on the distill corpus):

family model (d) lossless rank r/d recovery at the no-retrain floor
GPT-2 gpt2 (768) 16 2.1% rank-8 recovers 86%, then overshoots (overfit)
GPT-2 gpt2-medium (1024) 32–64 3–6% rank-32 ≈ lossless (+0.00)
GPT-2 gpt2-large (1280) 64 5.0% rank-32 +0.10, rank-64 −0.05
NeoX pythia-14m (128) 32 25% reaches lossless (tiny)
NeoX pythia-70m (512) > 128 — rank-128 only +0.00 (just reaching it)
NeoX pythia-160m (768) > 128 — rank-128 +0.24 (54–67% recovered)
NeoX pythia-410m (1024) > 128 — rank-128 +0.93 (recovery non-monotone)

Three descriptive readings (necessity-vs-method discipline — these are recipe statements, not lower bounds): (1) The #142 result generalizes across the GPT-2 ladder — retraining recovers the no-retrain truncation loss at every rung (54–211%), and the lossless rank stays a small fraction of d (r/d ≈ 2–6%), growing modestly with size (16 → ~32 → 64 over 768 → 1024 → 1280). (2) But it is NOT basis-robust across architecture at this budget — the GPT-NeoX (Pythia) models ≥ 70m do not reach lossless within rank 128 at 300 steps / lr 1e-4, where the GPT-2 models do (and recover far more per rank). So the NeoX disadvantage seen in frozen truncation (above) persists under retraining at this budget — it is not purely a frozen-linear artifact; whether more steps / a higher stable lr close the gap is open (the tiny pythia-14m does reach lossless, so the method itself works on NeoX). (3) “Lossless” here means matches base NLL on the distill corpus — the negative ΔNLL / >100% recovery at high rank is the bottleneck factors overfitting the small corpus, not free compression; a general-eval version is the natural follow-up. (runs/disassembly/core_distill_summary.json.)

…but compression is legibility-NEUTRAL — small ≠ legible for free (compress_legibility.py)

The feature-native endgame (train an N× smaller model → extract clean features / knowledge / circuits → the small Python program → runtime explainability) hinges on whether compression aligns with interpretability. It does not, for NLL-only compression: logit-lens-scoring the rank-32 directions the trained bottleneck writes into vs the untrained PCA directions, both interpretability proxies are unchanged — peak-z 5.00 → 4.96, closed-class fraction 0.10 → 0.09 (marginally lower, within noise). So distillation buys the size win (#142) but not a feature-cleanliness win: compression and per-feature monosemanticity are decoupled under an NLL objective. Consequence for the pipeline: to get both small and legible you must train with an explicit legibility / sparsity term (the sae-forge feature-native objective) — pure NLL-distillation packs the same entanglement into fewer dimensions. The enabling facts are all in hand (smaller = more decompilable; compression is tractable; feature/knowledge/circuit extractors + pylm + runtime idiom-traces exist), but the missing ingredient is the legibility objective, not more compression. (runs/disassembly/compress_legibility_summary.json.)

…and a naive legibility term can’t fix it on a tight bottleneck — the small / legible / complete trade-off triangle (feature_native_distill.py)

Adding the obvious legibility objective — an L1 sparsity penalty on the bottleneck codes (the sae-style term) — does not rescue legibility on a compressive rank-32 bottleneck: sweeping λ ∈ {0, 0.01, 0.05, 0.2}, peak-z stays ~4.95 and the code-active fraction stays ~0.50 (L1 doesn’t sparsify), while NLL only rises (+0.19 at λ=0.2). The reason is structural: a tight (rank < d) bottleneck has no room to be sparse — every direction is genuinely used, so L1 shrinks them all uniformly instead of zeroing any. Legibility needs overcompleteness — an SAE has more features than dimensions and then sparsifies, which is the room to specialize — and overcompleteness is the opposite of compression. So the user’s “train smaller and extract clean features” runs into the forge tax as a trade-off triangle — small / legible / complete, pick ~two:

corner small legible complete what it is
pylm ✓ (49 LOC + flat) ✓ (every token = a named idiom) ✗ (~half) the decompiled program
low-rank distill (#142) ✓ (~30×) ✗ (legibility-neutral) ✓ (lossless) compression
SAE / feature-native ✗ (overcomplete) ✓ (sparse monosemantic) ✓ the legible basis

The forge tax is this triangle: monosemanticity (overcomplete + sparse) and compactness (tight low-rank) pull in opposite directions, and the full-fidelity composition resists both unless you pay one of them. The whole pipeline (train smaller → extract features/knowledge/circuits → the small Python program → runtime explainability) is buildable, but it cannot be small and legible and complete at once — you pick the corner per use-case (pylm for an auditable runtime; distillation for a cheap one; an overcomplete feature-native model for an interpretable one). (runs/disassembly/feature_native_distill_summary.json.)

Building the legible corner — and pricing it (feature_native_sae.py). Training the overcomplete corner directly (an SAE on each layer’s update — MSE + L1, the standard recipe — vs the tight rank-32 PCA bottleneck) makes the size cost explicit. At 2×d (F=1536), the SAE is complete (99% variance, ΔNLL ≈ 0) and big (28 M params), but sweeping the L1 weight λ∈{.02,.1,.4,1} it never gets sparse or legible: L0 stays ≥ 0.30 (≥30% of features active) and logit-lens peak-z is flat (~4.8, below the tight bottleneck’s 5.0):

corner ΔNLL variance L0 (sparsity) size
tight rank-32 (small) +0.58 65% — (dense by construction) 0.6 M
SAE 2×d, λ=0.02 +0.01 99% 0.81 28 M
SAE 2×d, λ=1.0 +0.00 94% 0.30 28 M

Two structural reasons it won’t sparsify at 2×: the update is genuinely high-rank (rank-95 ≈ ⅓·d, from core_rank), so reconstructing it needs ~⅓·d active features — L0 can’t fall below ~256/F; and 2×d is not enough overcompleteness — real monosemantic SAEs run 8–64×d (jbloom’s GPT-2 SAE is 32×d = 24,576), where ~256/24,576 ≈ 1% can be sparse. So the size cost of legibility scales with the update’s intrinsic rank: the more entangled the core, the more overcomplete (bigger) the legible basis must be. That is the triangle’s deepest form — small/legible/complete trade against each other because the composition is high-rank, and the legible corner’s price (overcompleteness) is set by exactly the Θ(d) rank that the compressed corner squeezes out. (Proper monosemantic training at 8–64×d is the sae-forge sub-project; here the 2× run prices the corner and confirms the trade-off is structural, not a tuning miss.) (runs/disassembly/feature_native_sae_summary.json.)

The real legible corner, characterized — and it is not complete (legible_corner.py). A properly-trained 32×d SAE (jbloom GPT-2, F=24,576, resid_pre, all layers) is genuinely sparse (mean L0 0.025 — ~624 active of 24,576) and legible (decoder logit-lens peak-z 6.99, well above the 2×d SAE’s 4.8 and PCA’s 5.0) — confirming legibility needs the high overcompleteness #146 predicted. But composed across all 12 layers it pays the forge tax in full: the SAE-bottleneck generic NLL is 5.22 → 9.16 (Δ +3.94, near-random) — not complete. So the legible corner is legible + sparse + big but NOT complete: sparsity and completeness themselves trade off (the dense 2×d reconstruction of #146 was complete-but-illegible; the sparse 32×d is legible-but-forge-tax-incomplete). This sharpens the triangle — legible-AND-complete is itself blocked by the forge tax, because composing sparse monosemantic features across layers is exactly the composition that doesn’t factor through the feature basis. (runs/disassembly/legible_corner_summary.json.)

Minimum-to-run frontier — how small can the stored weights get (min_to_run.py)

How few stored bytes reproduce the model? Low-rank-factorize every attention + MLP weight matrix to rank r (embeddings left intact), and measure fidelity vs stored size. No-retrain SVD truncation is useless (the weights are not low-rank): even at rank 512 (89% of the weights kept) ΔNLL is +1.1 and agreement-with-the-full-model only 26%; at low rank it is near-random. But distilling the factors (init from SVD, train them ~400 steps, everything else frozen) recovers most of it:

GPT-2 weights (85 M) rank 8 (1% kept) rank 32 (6%) rank 64 (11%) rank 128 (22%)
no-retrain SVD ΔNLL +3.20 +2.54 +2.51 +2.40
distilled ΔNLL +0.54 +0.36 +0.26 +0.28

…because plain SVD is the wrong form, not the wrong rank — a data-aware truncation is far better, no retraining (min_to_run.py --data-aware)

Plain SVD minimizes weight-Frobenius error; what we actually care about is output error on the real activation distribution. A data-aware (activation-weighted / functional) low-rank truncation at the same rank — SVD in the data norm, --data-aware — is dramatically better, and the gap widens with rank, across a 6-rung ladder spanning both the GPT-2 and GPT-NeoX (Pythia) families (1 M → 302 M transformer-weight params; ΔNLL vs the full model, lower is better):

no-retrain ΔNLL rank 64 rank 256 rank 512
gpt2 (85 M) — plain SVD / data-aware +2.51 / +1.04 +2.18 / +0.11 +1.13 / −0.04
pythia-160m (85 M) — plain SVD / data-aware +5.17 / +1.37 +6.40 / +0.45 +4.43 / +0.07
pythia-410m (302 M) — plain SVD / data-aware +5.96 / +2.17 +8.86 / +0.76 +6.27 / +0.20
gpt2-medium (302 M) — plain SVD / data-aware +4.14 / +1.47 +3.77 / +0.46 +4.26 / +0.00

At rank 512 the data-aware truncation is 30–62× lower ΔNLL than plain SVD (and ~lossless on gpt2: −0.04 vs +1.13). Two consequences. (1) The apparent “low-rank floor” was a wrong-form result, not a wrong-rank one: the same rank-r subspace, chosen functionally, recovers far more of the model — so the frozen-SVD plateau understated compressibility. (2) The NeoX-vs-GPT-2 contrast is largely a form artifact. Under plain SVD, Pythia (GPT-NeoX) truncation is much more destructive than GPT-2’s (e.g. r=64: +5.17 vs +2.51) — but under the data-aware form the two families converge (pythia-160m r=256 +0.45 ≈ gpt2 +0.11), so most of NeoX’s “extra destruction” was the wrong norm on its basis, not an architectural fact. And the static data-aware truncation (no training) is already close to — and at matched rank sometimes beats — plain-SVD-with-distillation (pythia-160m r=256: data-aware-static +0.45 < the plain-SVD self-distill floor +0.65), i.e. a static functional diagnostic captures most of the achievable compression. (Descriptive, per the necessity-vs-method discipline: this says the diagnostic was weak, not that the function is small; “irreducible” claims still require a proof we don’t have.) Scope cap: the data-aware covariance accumulation OOM’d on the 8 GB GPU for pythia-1b and gpt2-large, so those two rungs have plain-SVD but not data-aware here. (runs/disassembly/min_to_run_summary.json, @svd vs @svd+daware tags.)

So with light distillation the composition weights factor to ~6–11% of their size for a model still capable on the corpus (ΔNLL +0.26 at rank 64 ≈ 9× weight compression). Two notes on scope: (1) this is a capable model, not a faithful copy of GPT-2 — faithfulness (top-1 agreement with the original) is a separate, harder axis: distilling the factors to the teacher (hard-label CE and soft-KL, 1500 steps, ranks 32–256) plateaus at ~34–38% agreement, flat across rank (6% → 44% of weights), recipe, and data — scaling the distillation corpus ~20× (12.8 k → 280 k tokens) and steps to 5 000 moved rank-64 only 34% → 38%. So the cap is not primarily data-starvation. The clean dissociation: the factored model is capable (NLL ΔNLL +0.26–0.64) but predicts different tokens than the original on the composition-dependent ~60% — a similarly-good different model, not a faithful copy. Careful framing (per the necessity-vs-method discipline): this is robust evidence that this approach — factoring all 48 matrices at one rank

Data scale and variety move faithfulness — and reveal it is content-dependent. Scaling the distillation corpus (single-domain Shakespeare 12.8 k → 280 k tokens) lifted rank-64 faithful agreement 34% → 38%; a diverse 5-domain corpus (drama + 3 novels + code) lifted it to 41% — monotone, no wall (so the cap is recipe/data-shaped, not a proven floor). Adding Wikipedia (6 domains) and breaking agreement down per domain exposes a huge spread the aggregate hid: austen 85% · shakespeare 34% · melville 33% · code 26% · shelley 25% · wiki 24%. The reading: a low-rank copy reproduces GPT-2 where its behaviour is retrieval / memorization (Pride & Prejudice — heavily-reproduced in WebText, near-deterministic — 85%) and fails where it is composition (Wikipedia facts, code structure — 24–26%). So the ~⅓ aggregate is a mix of near-fully-reproducible retrieval content and near-irreducible composition content — the same retrieval-vs-composition split, now measured by domain. (Confound: austen’s 85% conflates “memorized/predictable” with “stylistically self-consistent” — train and eval are the same novel — so the absolute spread over-states it; but the composition-heavy domains being hardest is the robust signal.) (runs/disassembly/min_to_run_summary.json.)

Variety helps only up to capacity — then it dilutes (at fixed rank). Pushing to 15 domains (12 novels across genres/eras + drama + code + wiki) at the same rank-64 / 12 k steps drops the aggregate to 30%, and the per-domain breakdown shows why: the copy faithfully holds only the heavily-memorized domains (austen 76% · stoker/Dracula 73% · dickens 46%) while the rest — and the composition-heavy ones especially — fall to 12–32% (kjv 12 · machiavelli 12 · wiki 16 · grimm 18 · code 20). So at fixed low-rank capacity, more variety spreads the model thinner rather than lifting it: capacity becomes the binding constraint across breadth. This says the variety push needs more rank in proportion to the breadth — consistent with the composition being Θ(d) and content-spanning — and reframes the earlier 34→41% climb as “variety helps until the rank-64 budget is spent.” (Still about this approach, not a floor.) Multilingual sharpens it: adding 6 non-English Wikipedia domains (rank 128, 21 domains) drops the aggregate to 21%, and the tail is dominated by the non-English domains — wiki_ru 0% · wiki_fr 4% · wiki_ja 5% · wiki_de/es 10% — because GPT-2 is out-of-distribution there: its non-English predictions are the least retrieval-like (no memorised non-English n-grams) and so the hardest for a low-rank copy to reproduce, while heavily-memorised English stays high (stoker 77 · austen 75). The reproducible-vs-not split tracks how in-distribution / memorised the content is.

A stronger teacher does not help a capacity-bound core — it hurts it (--teacher). Distilling a small core to a different, more capable same-tokenizer teacher (gpt2-medium core ← gpt2-XL, soft top-k KL, teacher pre-computed then freed so the student trains alone) is worse than self-distillation: XL-distill gives ΔNLL +3.4 (a bad model, ~30% agreement with XL) vs self-distill +0.77 (a good medium model, ~recovered). The medium architecture cannot represent XL’s distribution, so chasing its (unreachable) targets drags the core away from being a good medium model rather than transferring XL’s capability; raising the rank 256→512 nudged XL-agreement only 27→31% — “sizing the SVD for the teacher” within a smaller architecture can’t reach a bigger one. The minimum capacity to reproduce XL is bounded by XL’s own irreducible complexity (the forge tax): you can’t make a small model capable by distilling from a big one if it lacks the capacity. (A capability benchmark — benchmark.py, held-out perplexity + next-token + LAMBADA-style last-word accuracy — scores whether a produced core is good even when it isn’t a faithful copy.) (runs/disassembly/teacher_distill_summary.json.)

Capability ladder (the baselines). Scoring the full models on the benchmark (held-out perplexity · next-token · last-word, distinct from the distillation corpus):

model perplexity next-token last-word
gpt2 62.4 27.9% 26.5%
gpt2-medium 47.4 31.2% 27.5%
gpt2-large 41.6 32.8% 30.0%
gpt2-xl 38.7 33.8% 29.5%
Qwen2.5-1.5B 22.5 37.2% 37.5%
Llama-3.2-3B 18.9 39.6% 46.5%

The GPT-2 family improves smoothly with size (62→39 ppl), but the modern models are in a different regime — Qwen2.5-1.5B (22.5) and Llama-3.2-3B (18.9) roughly halve GPT-2-XL’s perplexity at similar/smaller size, with last-word accuracy 37–46% vs ~30%. This is the capability counterpart to the lower modern decompilable fraction (more capability ⇒ more entangled composition ⇒ less flat-reproducible) and the baseline any compressed core is judged against. (runs/disassembly/benchmark_summary.json.)

The clean Pythia ladder confirms the capacity-binding (controlled). The GPT-2 family confounds size with training recipe/data; the Pythia ladder holds those fixed (same Pile data, same GPT-NeoX architecture, same tokenizer — only width/depth differ), so it isolates size. (min_to_run.py now factors GPT-NeoX / nn.Linear weights as well as GPT-2 Conv1D — composition_mats factors the effective in→out map, Wᵀ for Linear.) At dims identical to the GPT-2 anchors — pythia-160m = 768/12 ≡ gpt2-small, pythia-410m = 1024/24 ≡ gpt2-medium — over 2500 steps at lr 1e-4: self-distillation recovers at every rung (pythia-70m ΔNLL +0.13, pythia-160m +0.63 at full rank), while a bigger teacher degrades a smaller student at every rank — pythia-70m←160m +3.9, pythia-160m←410m +4.0, and a two-rung gap (160m←1b) is no better (+3.9). The bigger teacher’s distribution is unreachable for the smaller student; chasing it drags it off being a good model of its own size — the same capacity-binding the confounded GPT-2 ladder could only hint at. Two method caveats logged (not findings): at lr 3e-4 the cross-teacher KL diverges (ΔNLL +169 — an optimization blow-up, not capacity; lr 1e-4 is stable), and GPT-NeoX no-retrain SVD-truncation is far more destructive than GPT-2’s at the same dims (+6.40 vs +2.18 ΔNLL at rank 256) — the same linear-low-rank form fits NeoX worse. (runs/disassembly/min_to_run_summary.json, keys pythia-*@self / @teacher=*.)

The low-rank basis was the weak link — not low-rank itself: functional (activation-weighted) SVD is near-lossless with no retraining (--data-aware). Plain SVD minimizes ‖W−AB‖_F, spending rank on weight directions the activations never visit; the model only cares about ‖X(W−AB)‖. Whitening each matrix by its input covariance C, taking the rank-r SVD in that norm (P = C^{-1/2}U plays U’s role, so the factor pipeline is unchanged), and unwhitening — no training — collapses the no-retrain error at the same rank: on pythia-160m, rank-256 ΔNLL +6.40 → +0.45 (14×) and rank-512 +4.43 → +0.07 (~lossless); on gpt2, rank-256 +2.18 → +0.11 (20×) and rank-512 +1.13 → −0.04 (denoises). The headline: data-aware no-retrain at rank 256 (+0.45) matches/beats the trained self-distill at the same rank (+0.65 after 2500 steps) — so the earlier “you must train the factors to move off the SVD subspace” was an artifact of the wrong norm; the right low-rank subspace is analytic, and 2500 gradient steps were largely rediscovering it. This sharpens (does not close) the open question: the residual floor — data-aware still wants ~⅓–½ rank for lossless — is the genuinely-composed core, and whether richer forms (cross-layer TT/MPS, a nonlinear bottleneck, or a sparse-overcomplete basis) beat that floor is the next test, rather than whether linear low-rank per se is too weak. (runs/disassembly/min_to_run_summary.json, keys *@svd vs *@svd+daware.)

Retraining the data-aware factors does not move the lossless floor to low rank — it shifts the capable-compression curve left ~3×. The full 2×2 ({plain, data-aware} × {no-retrain, 2500-step self-distill}) on pythia-160m, ΔNLL by rank: data-aware init gives retraining a large head start — data-aware+retrain reaches a given fidelity at ~⅓ the rank of plain+retrain (rank-64 +0.60 ≈ plain+retrain at rank-256 +0.65) — but the low-rank end plateaus capable-not-faithful: data-aware+retrain is +0.77 / +0.60 / +0.45 / +0.37 at rank 32 / 64 / 128 / 256, never approaching the +0.1 that data-aware no-retrain reaches only at rank ~512 (⅔d), and flattening by rank 256 (128→256 gains 0.08). So under this recipe the near-lossless floor stays ~⅔d — retraining stops the rank-waste but does not shrink the composed core. (Recipe-shaped, not a proven floor: a stronger recipe or a richer form — cross-layer TT/MPS, a nonlinear bottleneck, or a sparse-overcomplete basis — is the open path to beating ~⅔d.) (runs/disassembly/min_to_run_summary.json, keys *@retrain / @retrain+daware.)

Richer-form survey vs the linear floor — (1/3) a per-matrix nonlinear bottleneck does not help. First of the three richer forms (--nonlinear): an α-gated GELU bottleneck code + α·GELU(code), with α a learnable per-code-channel gate init 0 — a strict superset of the linear factor, so it starts exactly at the data-aware linear baseline and can only improve by adding nonlinearity. Data-aware-init + 2500-step self-distill on pythia-160m lands within noise of linear at every rank: ΔNLL +0.77 / +0.61 / +0.44 / +0.35 at rank 32 / 64 / 128 / 256 (vs linear’s +0.77 / +0.60 / +0.45 / +0.37); the α gate moves off 0 but nets nothing. So per-matrix nonlinearity is not the missing structure either — the floor is robust to {linear, nonlinear} × {Frobenius, data-aware} × {no-retrain, retrain}, i.e. to any per-matrix method. That points the remaining two tests (cross-layer coupling, sparse-overcompleteness) at structure that spans matrices — consistent with the area-law cross-layer entanglement the composition graph shows. (runs/disassembly/min_to_run_summary.json, key *@retrain+daware+nl.)

Richer-form survey — (2/3) cross-layer weight-sharing HURTS, monotonically (--share both / --share cp). Testing whether the layers share composition structure that per-matrix methods miss. Two joint factorizations of the layer stack, compared to independent per-matrix at matched stored size (pythia-160m, data-aware+retrain independent baseline): Tucker (--share both: shared in/out bases per matrix-type + a per-layer full core k×k) and CP (--share cp: shared rank-1 atoms read⊗write, each layer a per-atom scalar gate — i.e. the literal “cross-layer circuit” form, atoms capped at d so storage tops out ~11%). Trained 2500 steps, self-distill. ΔNLL at matched storage: independent +0.60 (11%) → Tucker +1.16–1.30 → CP +1.85–2.01 — a clean monotonic ordering: independent < Tucker < CP (the more sharing is forced, the worse it gets; init-robust — Tucker has a good HOSVD init). So the weight-level “few circuits” hypothesis is falsified: the layers’ composition weights occupy genuinely distinct subspaces; there is no compact shared cross-layer weight structure to exploit.

The resolution (and it reconciles this with the circuit catalog): two different objects had opposite answers. Weight-circuits — can the weight matrices be rebuilt from a few shared cross-layer atoms? No (this survey: high-rank, layer-distinct). Activation-circuits — do a few computational pathways carry the forward pass? Yes (the reconstruction-coverage milestone: induction / number-mover / …). These are consistent: a few sparse pathways threaded through high-dimensional, layer-distinct weight matrices. The computation is few-circuits (why the model reads as circuits); the weights are Θ(d) and distinct (why the size / forge-tax doesn’t compress). Net for the survey: the ~⅔d per-matrix data-aware floor is robust to {linear, nonlinear} × {per-matrix, cross-layer Tucker, cross-layer CP} — the composed core’s size is genuinely ~⅔d per layer and layer-distinct, even though its wiring is sparse. (Caveat: CP’s HOSVD-diagonal init is rough; but the monotonic ordering holds regardless, and even good-init Tucker loses to independent, so a better CP init would not overturn it. A targeted cross-layer form — adjacent-only coupling, not type-global sharing — remains untested.) (runs/disassembly/min_to_run_summary.json, keys *@retrain+share-both / +share-cp.)

Seeing the cross-layer channels — circuits teased from the activation coupling (circuit_channels.py)

The survey’s resolution (computation = sparse circuits, weights = high-rank/distinct) says circuits should be visible in the activation flow, which is low-dimensional. This isolates them: capture every layer’s residual write Δ_L, form the aggregate early write (layers ≤ cut) and late write (layers > cut) per token, unit-normalise each (directional coupling, else the last layer’s magnitude swamps it), and SVD the d×d cross-covariance C = Σ_p e_p⊗ℓ_p into channels (u_i early-dir, w_i late-dir). Each channel is then separately represented: writer/reader layer profile (directional, magnitude-normalised), logit-lens of u_i/w_i, and a firing pattern (per-position activation e_p·u_i, its duplicate-token enrichment and positional correlation). pythia-160m, cut 6/12:

So circuits are visible, teasable, and separately representable from the high-dimensional activations — correlationally. The honest limit shows up under causal ablation (--causal: project a channel’s direction out of the residual at the cut, so late layers can’t read it, and split ΔNLL by next-token category — punctuation / duplicate / other). Baseline NLL punct 2.51 / dup 1.94 / other 5.60. Ablating a single channel direction is NOT cleanly targeted: the dominant ch0 (88%) is catastrophic everywhere (ΔNLL +2.87 / +3.54 / +5.20 — a high-variance general axis, not a punctuation-specific circuit), while every small interpretable channel is diffuse and weak (mostly +0.04–0.2, “other” rising as much as “dup” — the induction-labeled channels do not specifically hurt duplicate-token prediction). So the layered answer to “can a circuit be separately represented?”: correlationally yes (decompose + label the coupling channels), but causally, as a single residual direction, no (one-direction ablation is global or diffuse — redundancy/distribution defeats it); causal circuits ARE real but live at the component (head/MLP) level, where targeted ablation works (the catalog: induction, the number-mover via agreement_circuit). The channel decomposition is a correlational lens on the coupling, not its causal factorization. Other caveats: aggregate-cut version (χ≈1.3, dominated by ch0; the per-layer-resolved core_mps coupling, χ≈16, would expose finer channels); some lenses are rare-token noise. (runs/disassembly/circuit_channels_summary.json.)

Did the catalog miss these — head-level circuits or known classes? (head_category_scan.py). Mean-ablate each attention head, split ΔNLL by next-token category (punct / dup / other). On pythia-160m: induction is a real localized head circuit — L1.H3 (dup ΔNLL +0.35, targeted +0.30) + L0.H1 / L2.H8 / L10.H5, confirming the catalog; punctuation / boundary has NO clean head (every effect ~+0.05, distributed) — so the dominant boundary channel is real correlationally but not a localizable circuit (we didn’t “miss” a clean one — the behaviour is diffuse); and copy-suppression (a known anti-induction class the catalog lacks) shows only a weak hint — L0.H11 ablation lowers dup-NLL (−0.09), the suppression signature, expected stronger on larger models (it was characterised on GPT-2 medium/large). Net: the grammar/boundary channels are distributed computation, not missed-localizable-circuits; copy-suppression is a genuine but faint catalog gap at this scale. (runs/disassembly/head_category_scan_summary.json.)

Are the channels topic broadcasts? (circuit_channels.py --diverse). A “topic broadcast” would be a channel set early and held / read across the document — so on a diverse corpus (Shakespeare / Austen / code / wiki) each channel gets a persistence (lag-1 autocorrelation, held-across-positions) and a register fraction (between-source variance, encodes which topic). Mostly no: the dominant + structural channels are per-token circuits — ch0 punctuation (register 0.04), the induction family (register ~0.05–0.12) — firing on local events identically across topics. But a minority are register/topic signals (high between-source variance = they encode what kind of text): ch5 microorganisms bacterial lipids (reg 0.45, science), ch2 LaTeX/code tokens (reg 0.43, code), ch10 multilingual scripts (reg 0.26). So topic-broadcasting is a real but minority phenomenon — the content/register channels, not the high-variance structural ones — and even those read as a register difference between docs more than a strongly held within-doc signal (persistence stayed ~0.2–0.3; none crossed a strong-broadcast threshold; 64-token chunks + 4 coarse sources, so a first cut). (runs/disassembly/circuit_channels_summary.json.)

Per-layer-RESOLVED channels expose the area-law bond (--resolved). The aggregate cut collapsed to one channel (χ≈1.3) because summing all early/late writes destroys the per-layer structure. The faithful version (core_mps-style: top-r PCA + standardise each layer’s write, then SVD the (early-layers·r)×(late-layers·r) cross-covariance into channels that mix layers) recovers it: on pythia-160m (r=24, cut 6/12) the coupling participation ratio is χ≈29 with a FLAT spectrum (top channels 6.7% → 1.7%, no dominant one) — the area-law structure, decomposed into ~20–30 roughly-equal channels. What they are: mostly the induction / copy machinery — nearly every channel is duplicate-enriched (dup×1.3–2.1) with writer layers early (0–6) → reader layers late (7–11), i.e. early-detect → late-act copy circuits; persistence is higher here (several at +0.41–0.47). And the register/topic broadcasts are sharper: ch3 microorganisms bacterial microbial bacteria (register 0.71, persistence 0.44 — a held science register, the clearest topic-broadcast), ch1 (register 0.72, byte/script tokens — a language/script register), ch8 (0.31). So the χ≈16–29 area-law bond decomposes into ~20–30 channels: dominantly induction/copy (early→late), plus a handful of genuine register/topic broadcasts — still a correlational lens (the causal caveat above stands). (runs/disassembly/circuit_channels_summary.json, --resolved.)

Where the incompressible core lives — diffuse content, not the circuits (core_residual.py)

If the ~⅔d size floor is caused by a few preservable circuits, exempting them would crack it. To check, data-aware-compress every composition matrix and split the compression ΔNLL by next-token category. pythia-160m (baseline punct 2.57 / dup 2.03 / other 5.63), ΔNLL at rank 64 / 128 / 256: punct +0.33 / +0.26 / +0.05 (boundary prediction compresses away easily — low-rank-friendly), dup +0.92 / +0.50 / +0.10 (the induction/copy family largely recovers by rank 256 — compressible), other (generic content) +1.66 / +1.02 / +0.48 (the residual that stays). The dup/other concentration falls 0.56 → 0.21 as rank rises: by the floor, the incompressible mass is overwhelmingly diffuse content composition, not the structural circuits. So preserving the copy circuits would not crack the floor — the irreducible part isn’t a handful of localizable circuits; it’s distributed content prediction. This is coherent with the rest of the survey (the channels were correlationally circuit-like but causally distributed) and locates the forge tax precisely: the entangled core is content composition — the “computed, not retrieved” remainder — which is why no low-rank or circuit-preserving weight trick shrinks it, and why the only route shown to break the frozen floor is learning (the #142 trained rank-8 update; the feature-native reconstruction). (runs/disassembly/core_residual_summary.json.)

What the content core COMPUTES — MLP-stored, context-bound, not attention-retrieval (content_mechanism.py)

Having located the forge tax on content prediction, the mechanism question (the “explain how we think” half): is that content in-context retrieval (attention-carried) or stored computation (MLP-carried)? Mean-ablate all attention vs all MLP, ΔNLL by category (pythia-160m, baseline punct 2.54 / dup 2.15 / other 5.71): attention +1.34 / +1.46 / +1.15 (≈uniform — structural routing), MLP +2.08 / +2.14 / +4.03 — MLP ablation hits content 2× harder than punct/dup. So content composition is MLP-carried, not attention-retrieval — the forge tax lives in the MLPs (computed/stored parametric), not retrieval-in-disguise. And it is genuinely context-bound: NLL on content drops monotonically 7.42 → 5.71 as the visible window grows k=1 → 64 (still falling at 64) — it needs the long context, not a local n-gram. Reconciliation: attention gathers the context, the MLPs compute the content prediction over it — the irreducible core is context-conditioned MLP computation. The open twist (next probe): MLPs read as key-value memories, so “MLP-stored” could still be flat-decompilable (a big KV lookup = the Θ(size) flat-knowledge term) rather than irreducible nonlinear composition — which is exactly the crux of whether the core unsquirrels into flat storage or stays computed. (runs/disassembly/content_mechanism_summary.json.)

The crux: the MLP content is DENSE, not a sparse key-value lookup (mlp_kv_sparsity.py)

MLPs read as key-value memories — if only a few neurons fire per token, the content is a sparse lookup (flat-decompilable, the Θ(size) term); if it needs most of the width, it is dense composition (the irreducible forge tax). Test: mask the post-activation MLP hidden to its top-k neurons per token at every layer, ΔNLL by category. pythia-160m (d_ff=3072, baseline content NLL 5.69), ΔNLL on content keeping top-k: k=64 (2.1%) +3.52, k=256 (8.3%) +2.97, k=512 (16.7%) +2.40, k=1024 (33.3%) +1.07 — still far from recovered keeping a third of the width. A sparse lookup would recover at small k; this does not. So the MLP content computation is DENSE in the native neuron basis, not a sparse KV lookup — the core does not trivially flat-decompile into a (big) lookup table; it is genuinely distributed computation.

This closes the decompilation arc: native-basis truncation (dense, here) and weight factorization (the whole survey, plateaus at ⅔d) both fail to sparsify/shrink the content core. The one route left — and now the indicated one — is a learned overcomplete basis (SAE / feature-native): because the native representation is dense, any sparsity must be learned, not found by truncation or factorization. (Framing: this shows the native basis is dense and truncation/factorization don’t sparsify it; whether a learned overcomplete basis does is genuinely open — achievability is not foreclosed, it’s the sae-forge sub-project’s bet.) (runs/disassembly/mlp_kv_sparsity_summary.json.)

Cross-model validation — the dense-content core is architecture-general and grows with scale

Re-ran all three probes on GPT-2 and the Pythia ladder (70m / 410m / 1b) to check the above isn’t a pythia-160m artifact. All three hold, and strengthen with scale: (1) residual = content — “other” is the largest compression residual at low rank with punctuation compressing first and dup/other falling with rank (gpt2 other +0.93 vs punct +0.59 @ rank 64; pythia-70m +0.99/+0.18; pythia-410m +2.27/+1.63 → dup/other 0.58→0.37; 1b OOM’d this probe — fp32 weight-clones + SVD don’t fit). (2) MLP-carried, not attention — MLP ablation hits content far harder than attention at every scale, and the gap widens: attention vs MLP ΔNLL on content = gpt2 +0.96 / +2.43, pythia-70m +0.69 / +3.13, pythia-410m +1.57 / +8.91, pythia-1b +1.70 / +8.13. (3) dense, not a sparse lookup — top-k neuron recovery of content needs ~50–67% of the hidden width on the small models and more at scale (pythia-1b: keeping 25% of d_ff still leaves +2.95 on content). So the entangled core — MLP-stored, dense, context-conditioned content composition — is architecture-general (GPT-2 and GPT-NeoX) and grows with model size, consistent with the Θ(d) forge tax; it was not a single-model artifact. (runs/disassembly/{core_residual,content_mechanism,mlp_kv_sparsity}_summary.json.)

Does the core grow faster or slower than the model? — FASTER, on both axes. The fraction of MLP neurons needed to recover content (top-k recovery, content ΔNLL < +0.3) rises monotonically with scale: pythia-70m ~48% of d_ff → 160m ~60% → 410m ~64% → 1b ~68% (equivalently, at a fixed 25%-of-width slice the leftover content error grows with size, 70m +0.96 → 410m/1b ~+3–4). So the dense content code is a growing fraction of an already-growing width — it outpaces the MLP width (compute axis). And the behavioral axis agrees: the pylm decompilable fraction falls 56% → 45% (pythia-14m → 1.4b), so the core’s behavioral share rises 44% → 55%. Both say the entangled core grows faster than the model — scaling does not dilute it, it concentrates it; bigger models are more irreducible, and the gap widens. This is the load-bearing tension for minimum-to-run: the flat-decompilable part (knowledge + structure) does not keep pace; the computed part outgrows it — and it is why the unsquirrel route had to be learned reconstruction (sae-forge), getting harder, not easier, at scale. (Caveat: the recovery-fraction is native-basis effective density — a learned overcomplete basis could shift the absolute numbers; the monotone trend is the result.) (runs/disassembly/mlp_kv_sparsity_summary.json.) Fitting the active dimension s (neurons to recover content at content ΔNLL < 0.3) as a power law in d across the ladder (scaling_fit.py): s ∝ d^1.23 (super-linear), so the active fraction s/m ∝ d^0.23 (robust: d^0.28 at threshold 0.5), and the superposition overhead over the ~d/3 functional rank, s/r, grows from 5.7× → 8.1× (∝ d^0.23). So “the core grows faster than the model” is a mild power law, not a step. (runs/disassembly/scaling_fit_summary.json.) Measuring the two inputs to that growth (feature_economy.py): the per-token “effective features in play” k (participation ratio of the MLP hidden) grows super-linearly, k ∝ d^1.15 (761→3731 across the ladder), while the functional overhead s/k ≈ 1.4 stays flat and the activation-covariance effective rank stays low (~50→94, ∝ d^0.46 — concentration stays low-dim, echoing χ≈16). So the active-dimension growth is driven almost entirely by more features active per token, not by a growing interference/packing overhead — s ≈ 1.4·k with k ∝ d^1.15. (runs/disassembly/feature_economy_summary.json.) Two follow-ups pin the rest. Entropy-dependence (entropy_dependence.py): binning tokens by next-token entropy H and measuring k, k is nearly flat in H — k ∝ H^0.03 (160m) / H^0.05 (410m) (corr 0.18–0.34). So the per-token feature count is a structural property of width, not per-token difficulty — i.e. the super-linear k growth is width-driven feature economy, not task-complexity. Packing efficiency η (sae_dictionary.py): a first pass at fixed sparsity L0=64 looked scale-stable, but that starved the big models (L0 ≪ k). The clean run scales L0 ∝ m (= 0.4m ≈ k per model) so reconstruction matches each model’s density; then the variance-explained gain from 0.5×m→4×m overcompleteness grows with scale (70m +0.028 saturating by 2×m · 160m +0.055 · 410m +0.074 · 1b +0.115 still rising at 2×m), i.e. the saturation width m′/m rises (2×→≥4×→≫4×) ⇒ η = m/m′ shrinks with scale — bigger models pack more total features per neuron. Crucially this does NOT contradict the flat s/k: the per-token active support s ≈ 1.4·k is k-driven (η does not enter, β≈0), while η governs the total dictionary size m′ (∝ d^{>1}), a separate quantity — the storage/legibility cost of a complete feature basis grows faster than the per-token compute. Net: the super-linear active-dimension growth (s ∝ d^1.15) is k itself (width-structural, entropy-independent), with a stable functional overhead s/k≈1.4; separately, the overcomplete dictionary needed for a legible basis grows faster than m (η↓). (Caveat: 1b’s absolute var (0.81) may be SAE-undertrained at d_ff=8192; the η↓ trend is clear already at 70m→410m.) (runs/disassembly/{entropy_dependence,sae_dictionary,sae_dictionary_cleanL0}_summary.json.) k is set at initialization, not learned (checkpoint_k.py): measuring k across Pythia-160m training checkpoints (step{N} revisions), k/m is flat from random init through training — step 0 (random) 0.438, step 1 0.438, step 1000 0.483, step 143000 (final) 0.410 (×0.94 overall, noise around ~0.42; the final value reproduces feature_economy’s 1261). So the per-token feature count is not accumulated by training dynamics — it falsifies the implicit-bias / “warm features” mechanism and shows k is a structural / statistical property of the architecture (the participation ratio of GELU activations at width d_ff), present in the random network. Therefore the “why k ∝ d^1.15” puzzle reduces to a statistics-of-GELU-at-width question (architectural), not a capability or learned-feature question — a much more tractable target. (The functional support s = top-k recovery is trained-dependent but tracks this geometric k as s ≈ 1.4·k in the trained model.) (runs/disassembly/checkpoint_k_summary.json.) The d^0.15 excess is from MARGINAL variance heterogeneity, not input covariance (init_statistics.py). Decomposing the architectural k: a pure-iid homogeneous-variance Gaussian → GELU gives k ∝ d^1.01 (linear, k/m ≈ 0.42 = the GELU constant c). Real activations give k ∝ d^1.16; column-shuffling the input (preserve per-coordinate marginals, destroy all cross-coordinate correlation) gives k ∝ d^1.23 — more super-linear. So the excess survives removing all input correlation → it lives in the marginal variance heterogeneity of the pre-activations (W↑x)ᵢ (the per-neuron variance spread, which grows with width), not the covariance structure. And cross-coordinate correlation reduces the PR (k_real < k_shuf), the opposite of a “correlation delocalizes” hypothesis. So the d^0.15 reduces to: the spread of per-neuron pre-activation variances grows with width, and GELU’s moment ratio (E|·|)²/E[·²] is σ-dependent, inflating the participation ratio super-linearly. (runs/disassembly/init_statistics_summary.json.) But measuring the variance dispersion directly (sigma_dispersion.py) corrects the natural next guess: the per-neuron σ² coefficient of variation decreases with scale (CV ∝ d^−0.79; the σ² participation-fraction rises 0.45→0.69), so the pre-activation variances get more homogeneous with width, not less. Yet the heterogeneity gain (k_shuf/k_iid) still grows — so the excess is driven by the σ-regime nonlinearity of c(σ), not the dispersion magnitude. And a formula check: the proposed closed form m·E[c(σ)] scales d^1.00 (linear — wrong); the correct vector-PR concentration limit (Σ E|hᵢ|)²/(Σ E[hᵢ²]) scales d^1.23 (overshoots); the actual per-token k is d^1.15, between the two — so finite-m correlations between |hᵢ| and hᵢ² matter and neither factorised form is exact. (runs/disassembly/sigma_dispersion_summary.json.)

Router-kernel prototype — the conditional-compute payoff with an init-computed budget (router_kernel.py). Putting the scaling result to work: the per-token active budget is architectural, B(d) ≈ 1.4·k(d), and k = the GELU participation ratio is computable from the model itself (or even random weights). The kernel (1) measures k → B → expert budget E = round(B·K/m); (2) trains the informed router (gate from [local input ⊕ early-layer global context], straight-through hard top-E over K static clusters) with the KL(full ‖ routed) objective at that fixed budget, base frozen; (3) reports KL-preservation at active fraction E/K vs a random-router control. Result: gpt2 E=40/64 (62% active MLP), KL(full‖routed) 0.38 (learned) vs 1.77 (random) — 4.6× better; pythia-160m E=37/64 (58% active), KL 0.70 vs 3.81 — 5.4× better. So the architecturally-determined budget + informed router preserves the full model far better than random routing at ~60% active MLP — “store the whole MLP, compute ~60% of it per token.” The ~40% saving is real but bounded by the dense content (k/m ≈ 0.4, so B/m ≈ 0.6); the KL is preserved-but-moderate at that budget (a higher budget or better router lowers it). (runs/disassembly/router_kernel_summary.json.)

Runtime (conditional) sparsity — is the dense content expert-sparse? (mlp_experts.py)

Storage-sparsity (low-rank, native top-k, an L1-SAE up to 43×d) doesn’t sparsify the MLP content. A different axis is runtime conditional sparsity (MoEfication): cluster each MLP’s neurons into K experts and per token only compute the active experts — you still store every expert, but run few. Test: cluster K=64 experts (k-means on the up-projection keys, so co-activating neurons group), route at the expert level (keep the top-E experts per token by activation mass), measure content recovery. Result (K=64, avg expert ≈48 neurons): content recovers (ΔNLL → small) only near E≈48/64 (75%) for pythia-160m and E≈32/64 (50%) for gpt2 — comparable to the neuron-level top-k (~60–67%). So this static MoEfication does not make the content expert-sparse; it is about as dense over experts as over neurons (the copy/dup family is somewhat more expert-localized). Caveat — this is the floor, not the ceiling: the clustering (key-direction k-means) and router (mass top-E) are naive; a learned router / co-activation clustering can concentrate routing far more (that is what a trained MoE / the feature-native model would learn). So conditional-compute remains an open lever — a static read-off doesn’t deliver it, but the learned version is untested. (runs/disassembly/mlp_experts_summary.json.)

A LEARNED router does deliver conditional sparsity — for GPT-2 (mlp_router.py, mlp_router2.py)

The static read-off (mass-routing) isn’t the only router. A learned gate, trained on the loss, is the open lever. First attempt (mlp_router.py: a per-layer linear gate from the local MLP input, soft-trained, hard-evaluated) underperformed static mass-routing — a soft-weight/hard-select mismatch plus a weak memoryless local gate. Fixing both along the lines the channel decomposition suggests (mlp_router2.py): (1) global context — feed every layer’s router an early-layer residual vector (carrying the register/topic broadcast + induction-detect state — gather early, route late); (2) straight-through hard top-E selection (train what you evaluate). The informed router then beats both the naive gate and static mass-routing at every E, on both models, and for GPT-2 reaches genuine conditional sparsity: content recovers at E≈8/64 (12% of experts, ΔNLL +0.36) — where static mass-routing needed ~50% and neuron top-k ~67% — and at 50% it denoises (−0.22). pythia-160m is harder: the informed router helps (~1.4 NLL better than mass at low E) but content stays dense (+2.7 at 12%), so its content is genuinely less routable. So conditional/learned routing — with a router informed by the broadcast/cross-layer structure — is a real lever the static decompositions missed (strong for GPT-2, partial for pythia-160m). Open next steps the structure suggests: a recurrent gate over the χ≈16 cross-layer bond, a hierarchical per-sequence topic gate, and band-shared routing across adjacent layers. (runs/disassembly/mlp_router_summary.json, runs/disassembly/mlp_router2_summary.json.)

The composition graph (mean-squared canonical correlation between layer-pair write coords) is densely coupled — every pair far above chance (0.34–0.56 vs 0.009) — with adjacent-layer coupling > distant (0.49–0.53 vs 0.34–0.37) and the strongest edges clustered at the output-assembly end (late-layer pairs) plus the embedding edge 0→1. A banded DAG over layers: local-dominant but globally coupled, condensing through the low bond.

Ontology (each layer’s top write directions typed grammar / content by logit-lens): grammar concentrates at the boundaries — the embedding end (detokenise into grammatical classes) and, most strongly, the output end (write the grammatical next-token classes) — and is sparse in the middle (content / compute). The U-shape is cleanest in gpt2-small (4 5 1 1 2 2 1 2 3 7 5 8); in larger models the output-end concentration dominates and the embedding-end grammar is partly masked by rare-/byte-token directions. So the typed graph: grammatical scaffold at the rim, entangled content composition in the core — the ontology of “what the core is made of,” per layer. (runs/disassembly/core_mps_summary.json.)

Where the recursive syntax lives — in the composition, not the basis (recursive_syntax.py)

The static basis holds only the categorial grammar (POS-class directions). The open question: is the genuinely Chomskyan part — hierarchy, long-range dependency, recursion — in that basis, or in the composition? The cleanest test is subject–verb agreement across intervening attractors (Linzen et al.): “The key near the cabinets is/are” — the verb must agree with the head (key, singular), not the nearest noun (cabinets, plural). A flat local program follows the nearest noun; true hierarchical syntax tracks the head across depth. Three read-only measurements:

attractor depth MODEL (full) MODEL, attention-ablated FLAT pylm (head / attractor)
0 (local) 100% (Δ+5.0) 98% (Δ+2.1) — number is on the last token 53% / 47% (no attractor yet)
1 100% (Δ+2.7) 0% (Δ−2.2) — follows the attractor 0% / 100%
2 97% (Δ+2.4) 1% (Δ−2.2) 0% / 100%
3 100% (Δ+2.0) 0% (Δ−2.3) 0% / 100%

(GPT-2 small; logit-diff Δ = lp(correct) − lp(wrong).) Three findings, all confirming the hypothesis:

This closes the grammar loop and ties it to the forge tax: the categorial grammar is in the static write-basis (decompilable, #133); the recursive/hierarchical syntax is in the composition (the entangled core the forge tax measures). “Simpler-than-Chomsky” in the basis, Chomskyan in the composition — demonstrated, not assumed. (runs/disassembly/recursive_syntax_summary.json.)

Which heads carry it — the number-mover circuit (agreement_circuit.py). Per-head ablation over the depth-≥1 stimuli localizes a small, distinct circuit (GPT-2, base agreement logit-diff +2.55): the load-bearing heads are 7.4 (ablation drop +0.85), 10.9 (+0.51), 8.5 (+0.50), 2.10 (+0.50), concentrated in mid-to-late layers. The late movers show the clean signature — they attend from the verb to the HEAD noun, not the nearest attractor (10.9: verb→head 0.31 vs verb→attractor 0.07; 10.5: 0.35 vs 0.06; 11.10: 0.29 vs 0.09) — literally moving the head’s number to the verb slot. There is also an early attractor-tracker (0.3 attends 97% to the last token; 2.10 leans to the attractor) — the local pathway the agreement heads override. Crucially, none of the number-movers are induction / prev-token / duplicate heads (all UNNAMED): the agreement circuit is a distinct operator class, a new catalog entry — the composition that does the recursive work, made of heads the copy-idiom catalog doesn’t name. Cross-scale: gpt2-large reproduces it — the movers are again UNNAMED mid-to-late heads (24.3 with verb→head 0.42 vs 0.03; 32.5, 25.4, 15.10, 14.0 at the same ~60–90% relative depth). (runs/disassembly/agreement_circuit_summary.json.)

The recursion depth limit — distance is interference-bounded, nesting breaks sooner, but the ceiling does not grow with layers (recursion_depth.py)

Is recursion depth-limited by the number of layers? Theory says a single forward pass is a fixed-depth circuit (TC⁰), so genuine nesting (each embedded clause resolved before the outer one — stack-like) is layer-bounded, while mere distance (the head noun is far away) is not — attention jumps directly across any distance in one layer. Testing both — PP-distance (“the key near the dogs near the tables is”) vs center-embedding (“the key that the dogs that the cat sees chase is”), scoring the outer verb’s agreement with the head across depth 0–5:

nesting ceiling (acc ≥ 75%) gpt2 12L Llama 16L gpt2-medium 24L gpt2-large 36L
distance (PP modifiers) 5+ 5+ 5+ 5+ (gradual, never crosses 0)
nesting (center-embedding) 5 3 3 2 (logit-diff crosses 0 at depth 5)

Honest conclusion: the TC⁰ layer-bound is real in principle but is not the binding constraint here. Every model has 12–36 layers — far more than the depth ~2–3 where center-embedding breaks — so they fail for distributional / interference reasons, not layer exhaustion: deep center-embedding is vanishingly rare in training (and unparseable for humans past ~2), each level adds more opposite-number signal (nouns and inner verbs), and a bigger/better model commits harder to the dominant local-natural parse → it breaks earlier. So the answer to “is recursion layer-limited?” is: yes in principle (one pass is TC⁰), but empirically these models cap at center-embedding depth ~2–3 — far below any layer limit — and layer count is not the active constraint; training distribution + interference are. The genuine layer-bound would bind only for in-distribution recursion pushed past the data’s depth, which natural language essentially never does (the human depth-2 center-embedding limit is itself this phenomenon); to go deeper a model uses the decode loop / chain-of-thought (TC⁰ per step, Turing-complete across steps), trading within-pass depth for sequence length. (Caveat: nesting adds more+stronger attractors per level than distance, so “nesting harder” is partly more interference, not purely stack depth; the robust, size-monotone result is the ceiling shrinking with capability.) (runs/disassembly/recursion_depth_summary.json.)

Does the decode loop buy back the depth? — chain-of-thought, partially (cot_depth.py)

Theory’s escape hatch: a single pass is TC⁰, but the decode loop is Turing-complete across steps, so chain-of- thought should trade within-pass depth for sequence length and lift the ceiling. Testing it on the canonical bounded-recursion task — Dyck bracket matching at nesting depth d (balanced vs one mismatched closer, chance 50%) — with an instruction-tuned model that can reason (Qwen2.5-1.5B-Instruct), direct answer vs CoT:

depth 2 4 6 8 10 12
direct 85% 65% 65% 65% 55% 50% (chance)
CoT 50% 85% 74% 50% 71% 43%

Net: the within-pass depth limit is real and clean (direct → chance), and the decode loop is a genuine but reliability-bounded escape — depth is traded for tokens, only as well as the model can actually reason step by step. (runs/disassembly/cot_depth_summary.json.)

Execution model: an interpreter over the op-graph (“ResidualVM”)

The recompile-KL harness is most useful not as a one-shot metric but as a steppable interpreter over the extracted op-graph: run the named ops on the residual bus, in selectable fidelity modes, and watch reconstruction-KL — which makes decompilation debuggable, not just measurable. (Execution-model framing contributed by Grok; integrated + corrected here.)

The honest mechanics — dataflow per pass, “VM” only at the loop. It is tempting to cast this as a von Neumann machine (fetch–decode–execute over a stored program). That is the wrong abstraction for a single forward pass: there is no program counter, no instruction fetched per cycle, no data-dependent control flow, and the weights are never modified by the data path. One pass is a fixed-depth dataflow circuit (closer to an ASIC / systolic array; Merrill’s TC⁰) — the “ISA” (the op-catalog) is hardwired and applied in parallel, a description of fixed functional units, not a runtime dispatch. The stored-program / von-Neumann character appears only at the autoregressive loop: the residual stream + KV-cache as a read/write tape, the decode step as the clock, chain-of-thought as working memory (see the llm-as-accreting-vm framing). So the interpreter executes a fixed circuit per token and is a VM at the generation level — not a stored-program CPU per layer.

Formal grounding (why the DAG/loop split is load-bearing, not decorative): the object is a clocked sequential machine = combinational core (the fixed DAG) + state register (KV-cache + the growing token sequence) + clock (the decode step). The DAG alone is weak — one bounded-depth pass sits in TC⁰ (Merrill–Sabharwal); the loop supplies the power — transformer + decoding + an unbounded scratchpad is Turing-complete (Pérez et al., CoT-expressivity). Two precisions: (a) the recursion goes through a discrete token bottleneck — high-dim state is sampled to a token and re-embedded, so depth-per-step is bounded but steps are unbounded (hard problems buy back missing within-pass depth with longer CoT); (b) the program is fixed, not self-modifying — the same DAG every step, only the data changes (microcode/ASIC-like; Turing power lives entirely in the outer tape+clock). The decompilation payoff: the loop is a clean recurrence — the hard part is reading the DAG, and reconstruction_coverage is exactly the measure of how much of that fixed high-dim DAG reduces to a compact symbolic program over explicit operands.

What the frame does buy, mapped to checked-in results:

VM concept what it actually is here grounded in
memory hierarchy (registers/L1 vs main memory) entanglement-tower levels: low-χ monosemantic core (addressable, cov95-high) vs high-χ entangled remainder mps_tower_*, serve-tower cov95-saturation-vs-capability-cliff
fidelity modes: full / preserve-hybrid / decompiled exact host / verbatim-pin ~6–12% of atoms + forge the rest / run only the recompiled op-graph preserve_hybrid_tiny.py, sae-forge NativeModel
ISA vs model-specific microcode the op-catalog (idioms) is invariant across architectures and languages; the sink/plumbing policy is model-specific the 4-model + multilingual results
associative memory bank MLP neurons as content-addressable key→value stores work-item 2 (the MLP gap)
the recompiler / JIT sae-forge projects the kept ops into a runnable module reconstruction-coverage metric above

What it adds beyond the metric (the real new lever): an interactive debugger. Step layer-by-layer; breakpoint when a named idiom fires; inspect the low-χ “registers” (SAE latents); ablate / preserve / swap a single op and watch reconstruction-KL move live. That turns reconstruction-coverage from a number into a tool for localizing where decompilation fails — i.e. it operationalizes milestones 1+4.

The hard constraint the frame must not hide. The entangled core is preserve-or-pay, not “approximate main memory”: by the tower’s no-go, fidelity modes move you along the interpretability↔capability frontier, never off it. An interpreter that “approximates the core cheaply” is just choosing a point on that frontier (and paying the capability cliff) — the ceiling is the forge tax, restated.

Demarcation: adopt the execution layer, the memory-hierarchy mapping, the fidelity modes, and the ISA-vs- microcode framing; drop the von-Neumann mechanics (no per-layer instruction fetch) in favor of dataflow-circuit-per-pass + VM-at-the-loop. COMPOSE / PRESERVE / TOWER_TRUNCATE are not opcodes the model runs — they are extracted descriptions (the DAG) or execution modes (knobs on the interpreter).

Milestones (each a PR, gated on the prior)

  1. Recompile-KL harness, built as the interpreter (§Execution model) — DONE (v1, GPT-2): scripts/disassembly/residual_vm.py (see First result). v1 recompiles by keeping ops at full fidelity and mean-ablating the complement; v2 = feature-basis recompilation via sae-forge NativeModel (the ceiling test, milestone 4). Now also a reusable ResidualVM class in that file — the consolidated, arch-generic intervention layer the ~20 disassembly scripts each re-implemented (load + corpus-mean + ablate_heads / ablate_mlps / patch_mlp / patch_attn / trace / attribution as composable context managers); python residual_vm.py --demo reproduces the induction reconstruction coverage through it. (The biggest tooling gap identified after the cross-model program: a single steppable debugger so an experiment is ~10 lines and the load/hook/merge bugs live in one place.) Feature-level interventions are now in too — vm.set_feature(layer, feat, target) / ablate_feature edit individual SAE latents (GPT-2 jbloom / Gemma Scope) directly in the residual; python residual_vm.py --feat-demo shows that ablating the subject’s dominant feature (a France/geography latent promoting “Alps”) at L5 of “ France” drops logp(" Paris") by ~9 nats — feature-native surgery, the rung from head/MLP-level interventions toward editing the catalogued features themselves.
  2. Composition-DAG extractor — weight-space edge scorer + path-patch gate; auto-recover induction + IOI; report new sub-DAGs. (Generalizes path_patch_induction.py / composition_graph.py.) DONE (GPT-2): scripts/disassembly/composition_dag.py — static composition predicts dynamic writer specificity (ρ=+0.37); the canonical induction K-chain and IOI Q-chain are auto-recovered AND live, imposters/random rejected (0% FP); 22 new live edges surfaced (see Composition-DAG section).
  3. MLP ops — neuron key→value catalog + named MLP idioms; add to the DAG + the recompile. DONE (GPT-2): scripts/disassembly/mlp_ops.py — the recompile now charges for MLPs (M1/bridge kept heads only); MLPs are load-bearing, concentrated in L0 (the detokenizer); head↔MLP composition edges are weight-legible (see MLP ops section).
  4. The ceiling test — reconstruction-coverage plateau vs the tower’s entangled core, same host; the unifying claim stands or falls. First result DONE (v2, tiny GPT): scripts/cov95_forge_tax/ceiling_test.py — content/factorability axes decouple; the capability plateau is GPU-scale-gated (see Ceiling test section).
  5. Cross-model — repeat the ceiling on Gemma-2 / Llama-3 / Qwen-2.5 (idea i) to test whether the decompilable fraction is architecture-invariant like the mechanisms are. DONE (op-selection ceiling, 5 models incl. the gpt2-medium control): scripts/gemma/cross_model_ceiling.py — the named circuit beats random everywhere (mechanisms invariant) but the decompilable fraction is not invariant; the gpt2-medium control disentangles it — the high fraction tracks the absolute-position family, not scale (see Cross-model ceiling section). The forge-basis ceiling stays SAE/GPU-gated for non-GPT-2.
  6. Executable decompilation & knowledge, across six models — DONE. The reconstruction-coverage idea (M1) run as a sufficiency test over GPT-2 ×3 + Gemma/Llama/Qwen: no small head-set is sufficient for induction (≤30% even under resample-ablation; seed-stable ±0–1%) and even IOI’s 26-head circuit isn’t, in isolation (reconstruction, substrate). Knowledge ported cross-model: ROME causal tracing recovers an architecture-invariant early-MLP-store → late-attention-readout flow, and the store is editable by activation patch (100% fact-transplant in 5/6, generalizing across relations; Gemma’s storage is distributed) (tracing, transplant). Nuance vs M5: the forge-basis decompilable fraction (M5) is absolute-position-family-specific, but the head-ablation reconstruction coverage / circuit distributedness (M6) tracks scale — different metrics, both true (scaling synthesis): small models are unusually localized, so a faithful decompilation is the budget curve, not a tiny op-graph.

First result (milestone 1, GPT-2)

residual_vm.py on GPT-2 (Shakespeare; floor = KL(host ‖ all-144-heads-mean-ablated) = 1.92): keep a head-set at full fidelity, mean-ablate the complement, sweep the budget B by marginal ablation importance vs a random control + the named induction circuit.

Composition-DAG extractor (milestone 2) — first result (GPT-2)

composition_dag.py unifies the two precursors into one extractor that reads the call graph, not one idiom. composition_graph.py gave the static adjacency (Elhage Q/K-composition on raw weights, mean-write removed) but only validated the single prev-token→induction K-edge; path_patch_induction.py gave the dynamic gate but measured an induction-specific collapse, so it could only confirm induction. M2 generalizes both: score the full K/Q composition DAG over all causal head pairs, then gate the strongest edges with an idiom-agnostic dynamic metric — the mean total-variation change in the reader’s attention pattern when the writer is removed from that port (ΔTV), defined for any reader. GPT-2, Shakespeare, weights + two forward passes; 170 gated edges (105 K / 65 Q); attention recompute is exact (max|Δ| = 9.85e-7 vs the model).

The one methodological move that makes this work: raw ΔTV grows with reader depth/magnitude, so a global random null is confounded (early-layer write-hubs dominate). The fix is a reader-matched null — for each reader, compare the real writer against random causal writers into the same reader head (the path-patching null that isolates writer specificity from reader depth). An edge is “live” if its ΔTV beats its reader’s matched 2σ null; specificity = ΔTV − matched null.

Feeds the recompile harness (milestones 1/4) — now tested, see the next section. The live edges this extractor confirms are exactly the keepable ops the ResidualVM reconstruction-coverage interpreter (residual_vm.py) should retain: M1 selects heads by marginal ablation importance, but the DAG supplies the structured op-set (which writer→reader wires carry the computation), so a DAG-guided keep-set is the natural upgrade to M1’s flat head-budget — and the recompile-KL then verifies the extracted sub-DAG executes. The M1↔M2 bridge below confirms this. Completing the DAG with V-composition + MLP nodes (milestone 3) is what makes that op-set whole.

Scope (honest). (a) The dynamic gate runs on natural text, so the IOI Q-edges are confirmed by generic attention reshaping (ΔTV), not the IOI-task logit-difference — that task-specific causal validation already lives in ioi_causal.py; M2’s contribution is the unified static→dynamic recovery. (b) ΔTV measures whether an edge reshapes the reader’s attention pattern (Q/K composition); V-composition (writing values without moving attention) needs a different readout, not done here. (c) MLP nodes are absent — they are milestone 3. Compute: GPT-2, CPU-feasible — weights + two forward passes over the corpus (one for behavioural labels, one for the path-patch gate); the full run is ~70 s wall-clock and ~5 GB RAM on CPU (no GPU), so it scales to any HF model the box can hold a forward pass of. runs/disassembly/composition_dag_summary.json (re-run the script to regenerate the figure).

Validating the new write-hub edges (follow-up to M2)

validate_new_edges.py takes the 22 new live edges to the next rung: a targeted single-edge path-patch with a behavioural readout (the induction-style strong test, generalized). For each edge A→B it surgically removes A’s output from B’s port, recomputes B’s attention, and measures the collapse of B’s named components — {prev-token, duplicate, induction, sink} — against a reader-matched random-writer null. A pattern collapse beyond the null names the edge’s function; ΔTV with no named collapse is real-but-unlabeled shaping.

Cross-model: the positional broadcast is GPT-2-specific (forward-pointer (c), done)

scripts/gemma/cross_model_positional.py runs the cross-model test. A key-only causal path-patch that respects each model’s RoPE is non-trivial to do faithfully across architectures, so the test goes to the representational question the broadcast hypothesis really turns on: is the prev-token head’s position carried in the key content (absolute, must be written into the residual) or in the QK rotation (RoPE, applied at attention time)? For each model it finds the top prev-token head, captures its pre-rotation keys over the corpus, and decomposes the key variance into the fraction explained by absolute position vs by token identity. The prediction: GPT-2 position-dominated (its keys must encode where — exactly what the sink heads broadcast), the RoPE models token-dominated (keys encode what; position is the rotation).

Method: a between-group variance decomposition on the head’s key vectors. position_fraction = the between-position-index variance (variance of the mean key at each absolute position) over the total key- covariance trace; token_fraction = the same over token identity (mean key per frequent token). Both are fractions of the same total, so pos/tok is a clean within-head ratio comparable across architectures.

model pos. encoding prev-token head key var: position key var: token pos/tok
GPT-2 absolute (wpe) 4.11 59% 18% 3.3
Gemma-2-2B RoPE 21.7 7% 23% 0.30
Llama-3.2-1B RoPE 0.2 2% 64% 0.04
Qwen-2.5-1.5B RoPE 13.4 9% 25% 0.35

CONFIRMED — a clean 1-vs-3 split (~10×). Only GPT-2’s prev-token key is position-dominated; every RoPE model’s is token-dominated. This is the cross-model explanation that ties the positional thread together: GPT-2 encodes absolute position as key content → its prev-token head must read that content → it depends on the early sink heads that broadcast it (the validate_new_edges collapse) → and GPT-2 is the only family member that depends on its sink (the sink-ablation result, position-independently = the absolute-positions signature). The RoPE models need none of this — position rides in the rotation — so they have no sink dependence and no positional-broadcast circuit. The figure (cross_model_positional.png, regenerable) shows the bars flip: position > token only for GPT-2. Note Llama-3.2-1B’s prev-token head is in layer 0 (head 0.2) and is the most token-pure of all (pos/tok 0.04) — it reads the raw token embedding directly and leans entirely on RoPE for position, the cleanest case of the RoPE pattern.

Scope. This is the representational confirmation (the key is position-encoded only in GPT-2), corroborating the causal GPT-2 result; a faithful key-only causal path-patch across RoPE models (forward-pointer (a)) is the heavier next step — now done (next subsection). Also next: re-run on an oracle-supervised host (#19/#20) — does training a more legible model shift the prev-token key’s position-vs-token content, i.e. does supervision touch the positional machinery or only the feature substrate? ~33 s for all four models. runs/gemma/cross_model_positional_summary.json.

Cross-model: the causal key-only path-patch (the heavier confirmation)

key_patch_cross_model.py does the causal complement #26 deferred. The representational test said GPT-2’s prev-token key carries position-content and RoPE’s doesn’t; the causal test asks: does removing an upstream head’s key content collapse the prev-token attention? It’s a forward-pass intervention so each model applies its own RoPE — for the top prev-token head B, replace B’s key input with norm(resid − A_out) (q untouched → key-only) for each upstream head A, and re-read B’s attention. RoPE rotates the patched key, so the relative match is preserved; only the key content changes. The intervention is exact: the zero-patch sanity is 0.00e+00 on every model (q/v untouched).

model pos. enc. prev-token head top key-patch collapser prev-token collapse robust z
GPT-2 absolute 4.11 sink head 1.3 −22% 118.8
Gemma-2-2B RoPE 21.7 5.4 (non-sink) −0% 4.8
Qwen-2.5-1.5B RoPE 13.4 0.0 (non-sink) −0% 1.8

CONFIRMED causally — GPT-2’s prev-token attention is carried by KEY CONTENT; the RoPE models’ by the rotation. Removing the sink head 1.3 from GPT-2’s prev-token key collapses its prev-token attention −22%, a huge standout (z 118.8 vs a median 0.1% over the other upstream heads) — and the collapser is a sink head, the positional broadcaster from #25. In the RoPE models, no upstream head’s key-content removal collapses prev-token (max −0%): their rotation, untouched by the content patch, still aligns q−1. This is the decisive causal pairing for #26’s representational split, and the third+fourth causal/representational signatures of GPT-2’s learned absolute positions (with the sink-dependence and the cross-model ceiling). For scale: the −22% comes from removing one upstream head’s key-content with the rest of the model fully intact — vs a 0.1% median over the other ~47 upstream heads (the figure key_patch_cross_model.png is the lone red GPT-2 bar beside the flat RoPE bars). The zero-patch sanity (0.0) is the numerical-fidelity check: norm(resid − A_out) reproduces the clean key exactly when A_out is forced to zero.

Scope. Llama-3.2’s prev-token head is in layer 0 (no upstream heads to patch), so the RoPE side rests on Gemma + Qwen — a real limitation for any model whose prev-token head sits at the very bottom (the patch needs an upstream writer). The patch is the direct A→B key path (resid − A_out at B’s layer). Next: the same key-only / value-only forward patch generalizes to any circuit — induction heads, the validated V-edges (#28) — to probe content-vs-rotation dependence beyond prev-token; and re-running it on an oracle-supervised host tests whether supervision (#19/#20) shifts the key-content dependence. key_patch_cross_model.py, runs/gemma/key_patch_cross_model_summary.json (~85 s, 4 models).

Generalizing the patch across circuits & channels — match (key) vs move (value)

circuit_content_patch.py runs the same faithful key-only patch on three circuits — prev-token (positional: attend to q−1), induction (content: attend to the key whose predecessor token == the current token), duplicate (content: attend to an earlier occurrence of the same token). The sharp question: is the key-content-vs-rotation split (#26/#31) about the architecture, or about the addressing type? RoPE’s rotation supplies position, never token content — so a content match must live in the key content in every architecture, while only positional matching can move to the rotation.

circuit type GPT-2 Gemma-2-2B Llama-3.2-1B Qwen-2.5-1.5B
prev-token positional +22% ✓ +0% skip (L0) +0%
induction content +17% ✓ +18% ✓ +70% ✓ +89% ✓
duplicate content skip (L0) +3% +9% +13% ✓

(✓ = the top upstream head’s key-content removal collapses that circuit ≥10% and ≥3× the upstream-head median; every zero-patch sanity is 0.0.)

GENERALIZED — key-content dependence is about the ADDRESSING TYPE, not the architecture. The positional circuit (prev-token) is key-content-dependent only in GPT-2 (the RoPE models read position from the rotation, −0%), but the content circuit (induction) is key-content-dependent in every model (+17…+89%): removing the upstream predecessor-writer from the induction head’s key collapses induction everywhere — the rotation cannot supply the predecessor token, so it must be in the key content. So #26/#31’s GPT-2-vs-RoPE split is specifically the positional register; the content instruction set (token-identity matching) is universal — exactly consistent with the mechanism-invariance results (induction is causally load-bearing in all four). A nuance: in GPT-2 the induction collapser is its prev-token head (4.11 — the canonical K-composition writer), but in the RoPE models it is an early head (their dominant prev-token head is late — e.g. Gemma’s is layer 21, downstream of its layer-4 induction head — so induction is fed by a separate early predecessor-writer); the universal fact (induction is key-content-fed) holds, the specific writer differs.

The move (value) channel — universal even where the key isn’t. The same forward patch runs value-only: feed norm(resid − A_out) to the reader’s value and measure the change in its OUTPUT (ΔV-out, the #28 readout). RoPE rotates Q/K but never the value — so what each circuit moves should be content-dependent in every architecture, even for the positional circuit whose key is rotation-only in RoPE.

circuit type GPT-2 Gemma Llama Qwen
prev-token positional 0.22 ✓ 0.05 skip 0.11 ✓
induction content 0.26 ✓ 0.12 ✓ 0.17 ✓ 0.24 ✓
duplicate content skip 0.19 ✓ 0.28 ✓ 0.12 ✓

(top value-patch ΔV-out; ✓ = >0.05.) The move channel is universal — value-content dependence everywhere, including prev-token (GPT-2 0.22, Qwen 0.11) whose KEY collapses only in GPT-2. So the architecture-specific positional register is confined to the key/match (score) channel; what heads move is content in every model because the value is never rotated. The value channel is also markedly more distributed than the key channel — the top value-mover is only ~2–3× the upstream median (vs ~10–100× for keys), so no single head dominates what a circuit moves (the redundancy theme — cf the V-edges adding nothing to the recompile keep-set, #30).

NET (the addressing register, decomposed): only positional matching is architecture-specific (GPT-2 puts it in the key content via the sink broadcast; RoPE in the rotation); content matching (induction/duplicate keys) and all moving (every circuit’s value) are universal — consistent with mechanism-invariance. Scope. Induction is the clean content circuit; duplicate’s readers are early/layer-0 (GPT-2 skips, noisier elsewhere); prev-token skips for Llama (layer-0 reader). Every zero-patch sanity is 0.0. circuit_content_patch.py, runs/gemma/circuit_content_patch_summary.json (~4 min, 4 models × 3 circuits × 2 channels).

Circuit-structured keep-set selection (M1↔M2 bridge) — first result (GPT-2)

dag_recompile.py closes the loop between the two milestones: it feeds the M2-extracted live sub-DAG into M1’s reconstruction-coverage harness (same mean-ablation metric, coverage = 1 − KL(host‖keep)/KL(host‖all-ablated)) and asks whether the weight-cheap DAG (weights + 2 forward passes) picks the keep-set as well as M1’s expensive marginal-ablation importance ranking (one forward pass per head). GPT-2, Shakespeare, floor KL 1.82; ~95 s on CPU (most of it the 144-head importance ranking the DAG sidesteps).

Takeaway. The bridge confirms the program’s central use of the extractor: a path-patch-confirmed sub-DAG is the structured op-set the recompiler should keep (rivals/beats greedy importance, ≫ random, no ablation sweep) — but the gate’s ΔTV is a circuit-liveness signal, not a drop-in importance score, so the keep-set must come from the confirmed circuit, not from thresholding raw connectivity. This is the M1→M4 hand-off: the DAG keep-set is what the feature-basis recompilation (milestone 4) should express in a clean basis.

Next: (a) feed validated new edges (single-edge path-patch of the write-hub candidates from M2) into this keep-set so the recompiler grows beyond the two textbook circuits; (b) make the gate jointly attention-liveness and output-importance (e.g. an attribution / logit-effect term alongside ΔTV) so connectivity becomes a true keep-set score; (c) re-run on an oracle-supervised host (#19/#20) — a more legible host should yield a cleaner live DAG and a higher-coverage keep-set. The script takes --dag-summary so any of these DAGs (other corpora, supervised models) drops straight in. runs/disassembly/dag_recompile_summary.json (re-run to regenerate the figure).

Adding the value pathway (V-edges) to the bridge

The bridge now also consumes the V-composition DAG (vcomposition.py, --v-summary). The K/Q circuit covers attention routing; do the composed-OV virtual heads (the layer-6 value-readers, induction content re-read as a value) carry reconstruction the K/Q circuit missed? We add the 22 live V-edge heads to the 12-head K/Q circuit and measure the incremental coverage against the same number of random and importance-optimal additions:

keep-set coverage Δ vs K/Q
K/Q circuit (12 heads) +0.333 —
+ 22 V-edge heads +0.338 +0.004
+ 22 random heads +0.412 +0.079
+ 22 top-importance heads +0.532 +0.198

No — the value-pathway heads are output-redundant. Adding the 22 V-edge virtual heads lifts coverage by only +0.004, less than adding 22 random heads (+0.079), and their marginal-importance ranks are mid-pack (23–134 of 144). So ΔV-out, like ΔTV, is an edge-COUPLING score, not head output-importance: the V-edges are a real composed-OV coupling (#28 — removing the writer changes the reader’s output), but those virtual heads are not load-bearing for the model’s next-token output (keeping them barely helps; the reader is driven by the writer yet itself contributes little to the logits). This unifies the bridge’s lesson across all three Elhage ports — K/Q (ΔTV) and V (ΔV-out) composition-coupling both fail to predict output-importance; the recompile keep-set is the path-patch-confirmed named circuit, full stop, not any coupling score. (Consistent with the program-wide redundancy theme: composition is distributed; a few named heads are load-bearing, the rest is redundant coupling.) runs/disassembly/dag_recompile_summary.json, value_pathway field.

MLP ops in the DAG + the recompile (milestone 3) — first result (GPT-2)

mlp_ops.py adds the COMPUTE instruction class. M1 and the bridge kept/ablated attention heads only — MLPs ran at full fidelity, so the coverage metric never charged for them — yet MLPs carry real computation (greater-than is MLP-dominated; mlp_catalog.py read the neuron key→value vocabulary). M3 extends the mean-ablation harness to keep/ablate MLP layers as well as heads (floor = all heads and all MLPs ablated), adds head↔MLP composition edges to the DAG, and names the load-bearing MLPs. GPT-2, Shakespeare, floor KL 3.39.

Scope (honest). (a) Mean-ablating all MLPs is severe (L0 dominates), so “attention-only / MLP-only” are necessity statements, not a clean attention-vs-MLP credit split — attention’s reconstruction value is the MLP-intact bridge (#23), where circuit heads reconstruct with MLPs on. (b) Static head↔MLP composition does not rank MLP recompile-importance (Spearman = −0.43): the most important MLPs are early (L0) and have the fewest incoming head→MLP edges (a depth confound), so the DAG edges give structure, importance comes from the recompile — the same lesson as the bridge’s “ΔTV ≠ KL-importance.” (c) The full per-neuron catalog + low-rank/named-idiom analysis lives in mlp_catalog.py; M3 reuses its read→write naming for the load-bearing layers only. (d) MLP→MLP dynamic gating remains future work (V-composition is now done — next section). runs/disassembly/mlp_ops_summary.json (re-run to regenerate the figure).

V-composition — the value pathway (completes the DAG edge types) — first result (GPT-2)

M2 scored K- and Q-composition and gated them with ΔTV — the change in the reader’s attention pattern. By construction ΔTV cannot see V-composition (Elhage’s third edge type): head A’s output feeding head B’s value changes what B moves, not where B attends. V-composition is how heads chain OV circuits (“virtual heads” / composed copies), so the DAG was incomplete. vcomposition.py adds it with the matching readout — static comp_V(A→B) = ‖OV_A·W_V^B‖/(‖OV_A‖‖W_V^B‖) (mean-write removed) + a dynamic ΔV-out: remove A from B’s value, recompute B’s output contribution (attention fixed), measure the relative change in B’s residual write; reader-matched null. GPT-2, faithful patches (recompute vs model 9.85e-7).

Scope (honest). (a) The readout is the change in B’s direct output (value patch, attention fixed) — the value analog of ΔTV, not a task-level loss metric. (b) The spearman_staticK_vs_dTV reported here (−0.17) is range-restricted (computed over only the top-static-K slice, a narrow static range) and is not the canonical K static→dynamic agreement — that is M2’s +0.37 over a broad top+random edge set; here the K-edges are only the dissociation control. (c) Q-value (Q-composition’s value analog) and MLP→head V-edges are natural extensions. The DAG now carries K, Q and V head-edge types. runs/disassembly/vcomposition_summary.json (~75 s on CPU; re-run to regenerate the figure).

Ceiling test (milestone 4) — first result (v2, tiny GPT)

ceiling_test.py recompiles the tiny GPT by forcing computation through the SAE feature basis (sae-forge NativeModel, native_in_basis), sweeping SAE width 1–8×, measuring three things at once: forged-model output faithfulness (KL vs host; unigram floor 2.17), feature-content retention (mAUC), and monosemantic factorization (cov95).

width forged-model KL cov95 host→forged mAUC host→forged
1× 43.3 0.60→0.00 0.92→0.67
2× 5.11 0.64→0.00 0.92→0.78
4× 5.08 0.64→0.12 0.93→0.84
8× 5.08 0.60→0.16 0.92→0.85

The single-ceiling prediction is refined into two axes — and the capability axis is scale-confounded:

So milestone 4 builds the harness and settles the content/factorability axis (decoupled, robust across widths) and identifies the capability-ceiling test as GPU-scale-gated — an honest partial result: the unifying “one ceiling” claim is wrong as stated (≥2 axes), and the clean capability test is deferred to better forge hardware. (The tower’s ~24% irreducible core is the target a GPU-scale capability curve would be compared against.) ceiling_test.py, runs/cov95_forge_tax/ceiling_test_summary.json.

Cross-model ceiling (milestone 5) — first result (5 models, abs-pos vs RoPE)

The forge-basis ceiling (M4) needs per-layer SAEs + sae-forge forging, which Llama-3.2-1B / Qwen-2.5-1.5B don’t have and which is globally broken at whole-model scale — so the cross-model ceiling goes to the op-selection metric, which is arch-generic: cross_model_ceiling.py runs M1’s reconstruction-coverage harness (coverage = 1 − KL(host‖keep)/KL(host‖all-ablated)) on five models — two absolute-position (GPT-2 144h, gpt2-medium 384h) and three RoPE (Gemma-2-2B, Qwen-2.5-1.5B, Llama-3.2-1B) — and compares the named induction circuit (prev-token + induction heads, found behaviorally) to an equal-size random keep-set. The gpt2-medium control is the key: it is larger than two of the RoPE models, so it separates scale from architecture-family. The question: is the decompilable fraction architecture-invariant like the mechanisms are?

model pos. enc. heads induction circuit random lift ratio
GPT-2 absolute 144 +0.202 (20% of pass) +0.049 +0.153 4.1×
gpt2-medium absolute 384 +0.197 (20%) +0.012 +0.185 16.9×
Gemma-2-2B RoPE 208 +0.031 (3%) +0.022 +0.009 1.4×
Qwen-2.5-1.5B RoPE 336 +0.090 (9%) +0.031 +0.060 2.9×
Llama-3.2-1B RoPE 512 +0.034 (3%) −0.000 +0.034 n/a

The decompilable fraction is not architecture-invariant — and the gpt2-medium control disentangles why: it’s the absolute-position FAMILY, not scale. The named circuit beats random in every model (lift > 0 → the op-catalog is real and the mechanisms are invariant). But its coverage-share splits cleanly by position-encoding: the absolute-position GPT-2 family reconstructs ~20% of the pass (4–17× random) regardless of size, while every RoPE model sits at 3–9% (1.4–2.9×). The control is decisive — gpt2-medium has 384 heads, more than Qwen (336) and Gemma (208), yet keeps the full 20% / 16.9× fraction the larger RoPE models lack. So the earlier GPT-2-smallest-and-only-absolute confound resolves in favour of family: GPT-2’s absolute positions concentrate the induction circuit into a few load-bearing heads; RoPE distributes it across many (more redundant), so a fixed named circuit explains proportionally less. The random-budget curves echo it: the absolute-pos curves rise with heads kept, the RoPE curves stay flat/declining (random head-sets add ~nothing). This is the same GPT-2-family-is-special pattern as the sink (only GPT-2 depends on it) and the positional-broadcast circuit (only GPT-2 has it) — three independent signatures of GPT-2’s learned absolute positions.

Scope (honest). (a) Op-selection ceiling (heads at full fidelity, complement mean-ablated); the forge-basis ceiling (M4) stays SAE/GPU-gated for non-GPT-2 — when per-layer SAEs land for the RoPE models it can be run apples-to-apples. (b) gpt2-large (720 heads) was tested and excluded — its 5-head circuit collapses to ~0% because the behavioral induction-head identification fails at that head count on the small eval (it picks a last-layer “inductor”); the disentanglement rests on gpt2-medium, where the circuit is clean. (c) The circuit is a fixed ~5 heads (behavioral); the size-controlled lift makes the conclusion robust to the exact heads (--n-prev/--n-ind expose it for sensitivity). cross_model_ceiling.py, runs/gemma/cross_model_ceiling_summary.json (~3 min, 5 models).

Reachability — host-width × oracle-supervision (first result)

The ceiling test asks what’s achievable; this asks what’s reachable by training. The entanglement-tower retrain no-go (“you can’t train interpretability in without losing capability”) used a reconstruction bottleneck — the mAUC axis that already survives forging, not cov95. host_width_sweep.py retries on the right axis: train tiny GPTs from scratch across host widths, with/without an auxiliary oracle-feature- recovery loss (a linear head from the residual must predict the exact per-token oracle labels), and measure native cov95 + capability.

host width params LM-loss unsup→sup cov95 unsup→sup Δcov95
32 1.7M 6.54→6.42 0.48→0.62 +0.14
64 3.4M 6.21→6.16 0.62→0.79 +0.17
128 7.2M 5.99→5.99 0.69→0.76 +0.07
256 † 16.0M 6.04→6.05 0.45→0.69 +0.24

† w256 is undertrained (16M params on 107k tokens; its unsup LM loss 6.04 is worse than w128’s 5.99) — its low unsupervised cov95 is a compute-budget artifact, not scarcity counter-evidence.

Feeds milestone 1: supervision yields cleaner, more monosemantic low-χ residuals — which are exactly the “registers” the ResidualVM interpreter reads, so a supervised host should decompile further (higher reconstruction-coverage at lower op-budget) than an unsupervised one. That’s the direct hand-off to the recompile harness.

What kind of pressure lifts cov95? (aux-mode comparison)

Is the lift real monosemanticity or just linear recoverability? monosemantic_aux.py compares aux modes at a fixed well-trained width (128), training from scratch, over 3 seeds (per-seed model init + batch order + eval-SAE init):

aux mode LM-loss cov95 (mean ± std, 3 seeds)
none 6.02 0.68 ± 0.04
linear (recoverability) 6.01 0.76 ± 0.00
decorr (orthogonal read-directions) 6.00 0.76 ± 0.00
dedicated (one-neuron-per-feature) 6.00 0.69 ± 0.00
sparsedict (full in-loop aligned SAE) 6.16 0.48 ± 0.07

Paired across seeds: linear > none in 3/3 (+0.080 ± 0.043); sparsedict < none in 3/3 (−0.195 ± 0.086); linear > sparsedict in 3/3 (+0.276) — every signal survives the noise.

The simple linear-recoverability proxy is robustly the best — and every “more direct” monosemanticity objective fails to beat it: orthogonalizing the probe’s read-directions is inert (decorr ≡ linear, identical every seed), dedicated raw neurons sit below none’s mean, and the full sparse-dictionary-in-the-loop (a jointly-trained TopK SAE on the residual, reconstruction + sparsity, first F latents aligned to the oracle) is robustly the worst — cov95 0.48 (below none in 3/3) and the only one with a real capability cost (+0.16 nats). The heavier the direct pressure, the worse it gets. So the cov95 lift comes from making features linearly prominent (so the downstream SAE can isolate them), not from forcing axis-alignment / sparse-coding in the host residual: the SAE does the factorization; an in-loop dictionary’s reconstruction pressure distorts the representation in a way that doesn’t transfer to the fresh eval SAE. This answers “is it just recoverability?” — yes, and recoverability is the effective lever for SAE-measured cov95. Bonus: supervision also makes cov95 variance-free (linear pins exactly 22/29 features over threshold every seed, vs none’s noisy 0.62–0.72) — it lifts cov95 and makes it reproducible. monosemantic_aux.py, runs/cov95_forge_tax/monosemantic_aux_summary.json.

Is the lift real monosemanticity, or the eval-SAE finding what we planted? (non-SAE cross-check)

cov95 fits a TopK SAE — so does the linear aux just make features recoverable in a way a linear-ish SAE prefers (circular)? legibility_crosscheck.py scores the none→linear lift in three bases (same symmetric-AUC scorer, only one involving an SAE), width 128, 3 seeds:

basis none → linear Δ (mean ± std) up in
sae (fitted TopK dictionary) 0.68 → 0.76 +0.08 ± 0.04 3/3
neuron (raw residual dims, no fit) 0.68 → 0.76 +0.08 ± 0.07 2/3
pca (rotated basis, fit without labels) 0.36 → 0.69 +0.33 ± 0.09 3/3

Corroborated — the lift is genuine monosemanticity, not circular. Both SAE-free metrics rise too: features become single-detector-isolable by raw neurons (+0.08) and, most strongly, by PCA components (+0.33). So supervision doesn’t merely make features recoverable in a way the eval-SAE prefers — it makes them axis-isolable in the natural and rotated bases as well. (For the unsupervised model, neuron-cov95 equals sae-cov95 exactly — the fitted SAE adds nothing over raw neurons here, so it can’t be inflating the result.) Mechanism: the biggest lift is in PCA — unsupervised pca-cov95 is only 0.36 (the oracle features sit off the high-variance axes), so supervision pushes the features into the residual’s principal (high-variance) directions, where any axis-aligned probe (neuron, PCA, or SAE) isolates them. That’s a basis-independent signature of real monosemanticity, and it explains how recoverability becomes monosemanticity: prominence = high-variance placement. legibility_crosscheck.py, runs/cov95_forge_tax/legibility_crosscheck_summary.json.

Caveats + scope: the aux loss pressures linear recoverability of the oracle (which, per the aux-mode comparison above, is the effective lever — direct decorr/dedicated/sparsedict objectives don’t beat it, and the in-loop sparse dict actively hurts). The scarcity trend is only cleanly visible in the well-trained regime (≤w128 here); confirming it at w256+ needs more tokens/steps (compute scaling), not more width. Tiny hosts, short training. Status of the open levers: the training-pressure axis is settled (decorr/dedicated/sparse-dict all tested, none beats linear-recoverability; multi-seed-confirmed) and the measurement worry is resolved (non-SAE bases — raw-neuron + PCA — corroborate the lift, see cross-check above → genuine monosemanticity). Remaining follow-ups: (1) multi-seed + adequately-trained wide hosts to clean the scarcity curve; (3) richer oracles (spaCy POS/NER) + curriculum annealing of the aux weight; (4) polygram geometry penalties. But the direction is clear and the cost is ~zero, so the reachability lever is real. host_width_sweep.py, runs/cov95_forge_tax/host_width_sweep_summary.json.

Does feature legibility buy circuit legibility? (oracle-supervised DAG)

The reachability lever makes the feature substrate more legible (cov95 up). oracle_supervised_dag.py asks the sharp follow-up: does it also make the composition more legible — or are knowledge (features) and computation (the DAG) the separate axes the forge tax says they are? Train tiny GPTs unsupervised vs oracle-supervised (the linear lever), 3 seeds, and on each model measure feature legibility (cov95) alongside circuit legibility (static→dynamic composition ρ; prev-token→induction recovery) and the prev-token head’s key position-vs-token content (the #26 probe), paired by seed.

metric none → linear seeds +
cov95 (feature) 0.690 → 0.736 (Δ +0.046) 2/3
induction-recovery (circuit) 1.18 → 1.35 (Δ +0.16) 3/3
static→dynamic ρ (circuit) 0.44 → 0.49 (Δ +0.04 ± 0.42) noise-dominated
prev-token key position fraction 0.33 → 0.13 (Δ −0.20) 0/3 (drops)

PARTIAL / SUBSTRATE-DOMINATED — largely separate axes. Supervision robustly reshapes what the residual represents: cov95 lifts, and the prev-token head’s key shifts token-ward every seed (Δpos −0.20) — the feature-recovery loss injects token/lexical content into the residual, which the key inherits, making it less positional. Its effect on the composition is at most marginal: induction-recoverability lifts a small but consistent +0.16 (3/3), while the broad static→dynamic agreement is noise-dominated (±0.42) on a 16-head host. So the lever acts mainly on the feature substrate, with only a small spillover to circuit recoverability — consistent with the program’s knowledge-≠-computation thesis (you cannot supervise circuits into existence with a feature loss). Scope (honest): the tiny 4-layer host is underpowered for circuit metrics — its induction is marginal (1.1–1.4× baseline) and its prev-token key is token-dominated even unsupervised (unlike GPT-2’s 4.11 at 59%), so the static→dynamic ρ is high-variance; a definitive circuit-legibility test needs a host with real circuits (GPT-2 scale), which the reachability lever can’t retrain on this budget. oracle_supervised_dag.py, runs/cov95_forge_tax/oracle_supervised_dag_summary.json.

Instruction reuse vs specialization — is the op-catalog an ISA? (the “LLM-as-VM” test)

The disassembly reads the named ops as a reusable instruction set. But are the same instructions recruited across different tasks (genuine reuse → one ISA), or does each task have dedicated heads (specialization)? We’ve shown the idioms are invariant across languages and architectures, never across tasks within a model — instruction_reuse.py is that test. It builds the head-class × task causal matrix: mean-ablate each named op-class and measure the damage to three distinct programs — generic LM (held-out next-token NLL), induction (NLL on the 2nd copy of a repeated random sequence), IOI (logit(IO)−logit(S)). A class “serves” a task if ablating it damages it beyond a random-head control.

The matrix is run with five tasks — generic, induction-copy, copy-names (a 2nd copy task, different content), successor (consecutive-number runs → increment), IOI:

op-class generic induction copy-names successor IOI serves
prev-token +0.01 +0.65 ✓ +1.69 ✓ +0.45 +0.16 copy family
induction +0.03 +6.53 ✓ +8.03 ✓ +5.46 ✓ +0.26 ✓ ALL 4 in-context
duplicate-token +0.05 +0.73 ✓ +7.46 ✓ +22.7 ✓ −0.05 copy + successor
name-mover −0.00 +0.10 +0.50 +0.42 +0.06 (self-repair)
S-inhibition +0.00 −0.10 −0.13 −0.19 +0.58 ✓ IOI
negative-mover +0.00 −0.27 −0.46 −0.32 −0.60 (writes against IO)

(values = ablation damage, + hurts the task; ✓ = beyond the random-head control.)

A reused low-level core + task-specific output heads. Thickening the matrix from 3 to 5 tasks flips the read toward reuse:

  1. None of the named ops are load-bearing for generic LM (every generic cell ≈ 0) — the catalog is in-context-task instructions recruited on demand, not an always-on ISA.
  2. The low-level copy/addressing ops are genuinely reused across the copy-family: induction is load-bearing for all four in-context tasks; prev-token + duplicate serve the copy+successor family. So the shared substrate ops transfer.
  3. Only the output head is task-specific: S-inhibition serves IOI alone.
  4. A bonus: successor is carried by the copy ops (induction +5.46, duplicate +22.7), not a dedicated successor head — the model increments partly by in-context copying.

So the named ops are a reusable instruction set for the shared substrate, composed into task-specific circuits at the output — closer to the VM “shared ISA + per-task programs” than the 3-task version suggested, but with the architecture-specific caveat that none are always-on. Caveat: name-movers read ~0 only because mean-ablation triggers the IOI self-repair (quantified in self_repair.py) — not genuine unimportance. instruction_reuse.py, runs/disassembly/instruction_reuse_summary.json.

Self-repair, made causal (self_repair.py). The name-mover caveat above is quantified: ablating the primary name-movers (9.6/9.9/10.0) drops the IOI logit-diff by −0.002 (the circuit looks robust), but ablating primaries and backups (9.0/9.7/10.x/11.2) together drops it −1.04 (+2.57 → +1.53). The backups are causally idle when primaries are present (Δ +0.26 from removing them) and carry the logit-diff once primaries are gone (Δ +1.04 — 4× larger). So the op is load-bearing, but a redundant backup pathway masks it under single-class ablation — a clean instance of the program-wide redundancy (the named circuit ships hot spares), and exactly why mean-ablation under-counts name-movers. runs/disassembly/self_repair_summary.json.

Does the in-context-copy op survive a non-attention mixer? — SSM vs attention sweep (data, not a verdict)

instruction_reuse.py found induction (in-context copy) to be the one genuinely reused low-level op across tasks. The disassembly reads that op as attention (a QK predecessor-match feeding an OV copy). Does anything like it appear when the sequence-mixer is not attention at all? Mamba is a pure state-space model — no heads, no QK match; mixing is a learned linear recurrence (a scan). ssm_induction.py runs this as an exhaustive sweep, reporting raw measurements rather than a clean conclusion: two families across three sizes each (Mamba 130m/370m/790m; GPT-2 small/medium/large), each measured for (a) induction GAIN = (1st-copy NLL − 2nd-copy NLL) on repeated random sequences, over 3 seeds × 3 context lengths; and (b) a per-layer localization profile = mean-ablate each layer’s mixer (Mamba mixer.out_proj / GPT-2 attn.c_proj), record the induction-NLL increase.

model mixer L hidden induction gain (μ±σ) gain @ len 12/24/48 top layer (depth, ΔNLL, conc)
gpt2 attention 12 768 +12.54±0.20 +12.8 / +12.5 / +12.3 L5 (d0.45, +0.90, c0.27)
gpt2-medium attention 24 1024 +12.45±0.21 +12.7 / +12.5 / +12.2 L3 (d0.13, +0.52, c0.21)
gpt2-large attention 36 1280 +12.38±0.18 +12.6 / +12.4 / +12.2 L0 (d0.0, +15.85, c0.92 — artifact)
mamba-130m SSM 24 768 +12.10±0.35 +12.4 / +12.2 / +11.7 L0 (d0.0, +8.73, c0.47)
mamba-370m SSM 48 1024 +12.33±0.27 +12.5 / +12.4 / +12.1 L0 (d0.0, +4.70, c0.38)
mamba-790m SSM 48 1536 +12.28±0.30 +12.6 / +12.4 / +11.9 L1 (d0.02, +4.43, c0.32)

What the data shows (descriptive):

  1. The in-context-copy GAIN is present and large in both families, in a narrow band — +12.10..+12.54 — across 6 models, 3 sizes/family, 3 seeds, 3 lengths. The behaviour that the disassembly attributes to attention is produced just as strongly by a model with no attention. The gain shrinks slightly with context length (12 > 24 > 48) in every model — a consistent secondary trend.
  2. The localization differs by family — but the headline number is confounded by coarse ablation. Mamba concentrates the effect on an early layer (L0/L1, large ΔNLL +4.4..+8.7) plus a mid-deep secondary peak (L17/L34/L23, visible in runs/gemma/ssm_induction.png); GPT-2 small/medium distribute it across early-mid layers with small per-layer magnitudes (concentration 0.27/0.21). gpt2-large’s L0 spike (+15.85, conc 0.92) is an artifact: mean-ablating L0 in a 36-layer model destroys the detokenizer, not “induction” — so the auto-summary’s “attention concentration 0.47” is dragged up by that one outlier (gpt2/medium are 0.21–0.27). Within Mamba, concentration falls with size (0.47 → 0.38 → 0.32).

What is NOT concluded (the held-off pattern). This is evidence the in-context-copy capability is architecture-invariant — it does not show the mechanism is the same. (1) the GAIN is a behavioural NLL effect; (2) whole-layer mean-ablation removes everything a layer does, an over-estimate of “induction localization”; (3) the two families’ baselines/magnitudes are not directly comparable — compare within family across size, not across families by magnitude; (4) single seed for the localization profile; (5) the SSM has no heads, so there is no head-level resolution — we cannot point to a “Mamba induction head,” only a layer. So: the copy op survives the loss of attention behaviourally, and even survives the loss of heads as a unit of analysis; whether the SSM implements it by anything resembling a QK-match→OV-copy is unverified here and would need an SSM-native circuit probe (the recurrence’s state-update, not a head). This complements the cross-model ceiling result (mechanisms invariant, positional register absolute-position-family-specific): here the mixer class itself changes and the copy capability persists. ssm_induction.py, runs/gemma/ssm_induction_summary.json (~SSM models run on the sequential Mamba kernel — slow but exact).

Deep per-operator dossier — the full battery on ONE instruction (induction, first)

The disassembly studied each named op piecemeal: induction lived in composition_dag/ssm_induction, prev-token in cross_model_positional/key_patch_cross_model, name-movers in self_repair, the key/value channel in circuit_content_patch, reuse-across-tasks in instruction_reuse. operator_dossier.py inverts that: pick one operator and run every measurement on it in a single report — --op induction (default), extensible to prevtok|duplicate|name_mover|s_inhibition. Six sections (A identity · B causal×tasks · C K/V channels · D composition in/out · E redundancy curve · F cross-model). First dossier — induction on GPT-2:

Net (descriptive): consolidating the scattered measurements onto one operator reproduces each prior thread and surfaces what the piecemeal layout obscured — the head-set sensitivity of the causal profile (B) and the superadditivity of the redundancy (E). SAE-feature operands (what induction reads/writes in feature space — the “subword name-completion” finding) are the one battery section not run here (needs a SAE); flagged as the next layer. operator_dossier.py --op induction, runs/disassembly/operators/dossiers/induction/gpt2.json. The harness runs the same six sections for the other operators.

The operator CATALOG — every operator class, surveyed across every model

The dossier goes deep on one op; the catalog goes wide and turns the survey into a browsable artifact (docs/operators/, tree under runs/disassembly/operators/). Two instruments: operator_atlas.py (the cross-model matrix) + operator_dossier.py (the deep per-op pages), stitched into docs by operator_catalog_doc.py. The key framing the survey forced: each “operator” is a CLASS — a family of heads — not a single head. The catalog reports per-(class, model): behavioural signal (max head mass on the op’s pattern), membership (# heads carrying it), top head + depth, and a uniform causal ΔNLL (mean-ablate top-3, generic prose).

The catalog matrix — 7 universal/addressing operator classes × 6 models (GPT-2 small/medium/large + Gemma-2-2B / Llama-3.2-1B / Qwen-2.5-1.5B). Signal (membership in parens for GPT-2):

class kind gpt2 -medium -large gemma-2 llama-3.2 qwen-2.5
prevtok positional .96 (31h) .99 .96 .84 .68 .77
induction content .92 (22h) .91 .96 .94 .94 .99
duplicate content .62 (4h) .86 .96 .85 .74 .97
sink addressing .95 (117h) .96 .96 .07 (0h) 1.00 1.00
self addressing .83 (10h) .45 .55 .95 .96 1.00
local positional .34 (16h) .34 .34 .31 .27 .28
structural structural .19 (2h) .25 .38 .23 .36 .16

What the survey shows (descriptive):

  1. Induction is the most universal class — signal .91–.99 in all six (the content-copy op the SSM port + cross-model ceiling already flagged as architecture-invariant). prev-token and duplicate are present everywhere too (prev-token weaker in RoPE, .68–.84).
  2. The sink is the one class that splits by architecture — and it splits 5-vs-1, not GPT-2-vs-rest: present in the GPT-2 family (.95–.96, 117–553 heads) AND Llama/Qwen (1.00, 292–472 heads), but absent in Gemma-2 (.07, 0 heads). So “attention sink” is common but not universal; Gemma trained without one. And it is not load-bearing on prose anywhere (causal ΔNLL ≈ 0 every model) — present ≠ depended-on (the sink_ablation magnitude-vs-dependence point, now in the matrix; GPT-2’s deeper dependence needs the blocked-attention probe, not mean-ablation).
  3. RoPE models lean on self where GPT-2 leans on the sink — self signal 1.00/.96/.95 (RoPE) vs .45–.83 (GPT-2), and self is the only universally load-bearing class on prose (causal ΔNLL Qwen +3.77, Gemma +0.51) — the diagonal/current-token read is a RoPE workhorse.
  4. structural is sparse (1–7 heads) but occasionally load-bearing — Llama’s lone structural head ΔNLL +1.47.

The deep dossiers — 9 GPT-2 operator classes (docs/operators/<op>.md): the 7 idiom/addressing classes behaviourally found + the IOI circuit classes (name-mover, backup name-mover, negative/copy-suppression, S-inhibition, coreference), which are GPT-2-only (literature DLA head-sets, no published set in the RoPE models) so they sit outside the cross-model matrix. Highlights the catalog surfaced: the sink “class” is largely content heads in their idle state (its top-mass heads are 5.1/6.9/7.10 — the induction heads — so ablating “sink” hurts induction +4.6, not generic); copy-suppression ablation raises IOI (−0.54, it writes against IO); backup name-movers serve successor; and the head-set sensitivity the piecemeal scripts hid (the behavioural induction top-5 omits 7.11, so it does not damage successor where instruction_reuse’s literature set did +5.46).

Taxonomy (answering “N operators or N classes?”): classes. (1) class = the operation (these rows); (2) instance = an individual head (the disassemble_gpt2.py per-head listing; each dossier §A lists members); (3) variant = intra-class structure (induction writer-branching; token vs subword-name-completion inductors; sink-as-idle-state). Documented gaps: succession/greater-than is MLP-dominated (no clean attention head — carried by the copy ops); SSM/Mamba has no heads (induction present only behaviourally, ssm_induction.py). Not yet run: the SAE-feature operands per class (needs a SAE); the deep dossier §C/§D/§E ported to the RoPE models (the atlas covers them at class level; full-depth cross-model is the next layer). operator_atlas.py, operator_dossier.py, operator_catalog_doc.py; docs/operators/; runs/disassembly/operators/atlas_summary.json.

The CIRCUIT catalog — composed circuits, surveyed & collected across models

Operators are single head-classes; circuits are their compositions (a writer-op feeding a reader-op’s K/Q/V port, chained). The same cataloging machinery applied one level up: circuit_atlas.py (cross-model edge liveness + harvest of the GPT-2 discovery artifacts) + circuit_catalog_doc.py → docs/circuits/. The primitive is the edge (writer-op → reader-op via a port); the discovery gate (composition_dag ΔTV path-patch vs a reader-matched null) is what collects them. 7 circuits catalogued from two sources.

Cross-model circuit-edge liveness (remove the writer from the reader’s key → % attention collapse, 6 models):

circuit defining edge gpt2 -medium -large gemma-2 llama-3.2 qwen-2.5
induction prev-tok head –K–> induction +17% +23% +8% +18% +70% +89%
positional_broadcast sink/hub –K–> prev-tok key +22% +32% +0% +0% (skip) +0%
duplicate same-token reader (writer often L0) (skip) +0% +4% +3% +9% +13%
  1. The induction edge (prev-token → induction) is live in every model — and stronger in RoPE (Llama +70%, Qwen +89% vs GPT-2 +8–23%): content matching lives in the key everywhere, so removing the predecessor-writer collapses the induction reader’s attention universally. (Cross-model mechanism-invariance, at the edge level now, not just the node.)
  2. positional_broadcast is GPT-2-small/medium-only (+22%/+32%, ≈0 in gpt2-large/Gemma/Qwen, skip Llama) — the same absolute-position signature as the operator catalog’s sink: RoPE reads position from the rotation, so the prev-token key has no upstream writer to remove. The decompilable plumbing circuit is family-specific even though the content circuit is universal.

GPT-2 discovered / circuit-specific (harvested from the committed discovery runs, no recompute — GPT-2-only): the IOI Q-chain duplicate→S-inhibition→name-mover (5 Q-edges live; causal z from ioi_causal: negative/ copy-suppression z=62 writes against IO, duplicate z=6.0; self-repair: −primaries ΔLD −0.002 but −both −1.04, backups are hot spares); the V-composition “virtual heads” (induction 5.9 → layer-6 value 6.7, ΔV-out 1.32 — composed-OV, changes what is moved); the induction K-chain weights (canonical writer 4.11, 4/5 edges live); and the 22 NOVEL-LIVE discovered edges (13 behaviourally named — mostly early sink/write-hub → prev-token-key broadcasters: the collection-goal output, validate_new_edges).

Taxonomy & gaps. Levels: circuit (a DAG of operator nodes) → edge (writer→reader via a port) → operator class at each node. Gaps: succession/greater-than is MLP-dominated (no clean attention-composition circuit); SSM/Mamba has no heads (no composition edges; induction only behavioural). Not yet run: the IOI Q-chain / V-composition cross-model (no published head-sets off GPT-2); per-edge path-patch of all 22 discovered edges on the RoPE models. circuit_atlas.py, circuit_catalog_doc.py; docs/circuits/; runs/disassembly/circuits/atlas_summary.json.

ResidualVM debugger — a programmatic discovery engine (the catalog’s growth tool)

The catalogs above are surveys of what we named; the debugger is the tool that finds what we have not. It is programmatically steppable (not a human REPL — it returns structured data; a UI could sit on top): it instruments the forward pass as a steppable interpreter and exposes trace (per-head/MLP residual writes), intervene (mean-ablate any heads/MLPs → next-token KL), logit_lens_step (per-layer KL-to-final: where the answer is decided), attribution_sweep (ablate every head + every MLP one-at-a-time, rank by causal effect on a behaviour, and flag the strong components the catalog has NOT named = candidate new operators), and edge_probe (path-patch each upstream component out of a reader → candidate circuit edges). main() drives it; it is the engine for being exhaustive, not a place to hand-poke.

Run on GPT-2 across three behaviours (induction / IOI / generic LM), it immediately produced new catalog work:

So the debugger closes the loop the goal asks for: it generates candidate operators (7.6, 5.9, 8.3, MLP0) and candidate circuit edges (2.11→7.6, 5.9→7.6) that are not yet in the catalog — the next things to dossier. residual_vm_debugger.py, runs/disassembly/residual_vm_debugger_summary.json (GPT-2; intervention harness is arch-generic, the QK edge-probe is GPT-2-specific here).

The other instruction class — MLP / COMPUTE, across models

Attention MOVES operands; the MLP COMPUTES on them — and a salient discovery-sweep result was that MLP0 had the largest single-component causal effect measured. The operator catalog was attention-only; mlp_atlas.py adds the COMPUTE class, surveyed across architectures: per (model, layer) it mean-ablates the whole MLP block and reads the causal ΔNLL on generic prose + induction. (Mamba/SSM is excluded — no separate MLP block, the COMPUTE analog of “no attention heads”.)

model L all-MLP ablated ΔNLL (generic) top generic MLP (depth, Δ) top induction MLP (depth, Δ)
gpt2 12 +2.09 L0 (d0.0, +1.70) L0 (d0.0, +11.72)
gpt2-medium 24 +2.69 L0 (d0.0, +7.32) L0 (d0.0, +20.94)
gpt2-large 36 +5.28 L0 (d0.0, +3.67) L0 (d0.0, +13.57)
gemma-2-2b 26 +10.74 L25 (d1.0, +0.84) L0 (d0.0, +4.25)
llama-3.2-1b 16 +4.18 L1 (d0.07, +7.35) L1 (d0.07, +12.65)
qwen-2.5-1.5b 28 +4.29 L1 (d0.04, +7.64) L2 (d0.07, +13.91)

What the survey shows (descriptive):

  1. COMPUTE concentrates on an early MLP — the detokenizer — in 5 of 6 models (GPT-2 family L0, Llama L1, Qwen L1–L2): the single biggest COMPUTE op sits at depth ~0, and its ablation is catastrophic for induction (+11.7…+20.9 ΔNLL — far above any single attention head, confirming the discovery engine’s MLP0 finding).
  2. Gemma-2 is the exception — distributed COMPUTE: no single MLP dominates (top generic only +0.84), yet the whole-stack ablation is the largest of all (+10.74). Its COMPUTE is spread across layers, not concentrated in a detokenizer. (A second cross-architecture split, alongside Gemma’s missing sink in the operator catalog.)
  3. The whole-MLP-stack ablation is large in every model — COMPUTE is load-bearing everywhere, unlike any single attention-op class (no named attention op damages generic LM; the MLP stack always does).

GPT-2 deep characterization (harvested): the COMPUTE vocabulary is low-rank (mlp_catalog.py: transform participation 22 vs 1666 random — a small reused set of read→write templates, heavier-tailed than attention’s ~5); MLPs carry the reconstruction coverage (mlp_ops.py: MLP-only coverage +0.46 vs attention-only ≈0 — they interact); head↔MLP composition edges exist in weight space. Per-neuron read→write idioms are GPT-2-only (the cheap token-unembedding basis); the cross-model rows are per-layer causal profiles. mlp_atlas.py, runs/disassembly/operators/mlp_compute_summary.json, docs/operators/mlp_compute.md.

Boundaries / risks

Dependencies (cross-repo)

sae-forge (recompile/verify — the metric), polygram (operand-dictionary geometry — the basis), n-orca (emit the circuit DAG as a typed graph), larql (decompile→query→recompile as a queryable index). The disassembly toolkit here (scripts/disassembly/, scripts/gemma/) is the front-end that produces the ops.