Circuit catalog — composed circuits, surveyed & collected across models

Operators are single head-classes (../operators/); circuits are their compositions (a writer-op feeding a reader-op’s K/Q/V port, chained). A working catalog (amateur, exploratory, provisional — not a definitive reference) of 7 circuits collected with the tools here, two sources:

• Cross-model circuit edges — the defining composition edge of each universal-reader circuit, path-patched across 6 models (faithful key/value patch, arch-generic).
• GPT-2 discovered / circuit-specific — harvested from the committed discovery artifacts (composition_dag, vcomposition, ioi_causal, validate_new_edges): the IOI Q-chain, the V-composition virtual heads, and the 22 novel-live edges the discovery gate found (of which 13 are behaviourally named). These are GPT-2-only (literature IOI head-sets / GPT-2 path-patch runs).

Circuit inventory (index)

• induction — cross-model · also an operator (circuit:induction here vs op:induction there)
• positional_broadcast — cross-model
• duplicate — cross-model · also an operator (circuit:duplicate here vs op:duplicate there)
• ioi_q_chain — GPT-2 (Q-composition chain (GPT-2-only))
• induction_kchain_weights — GPT-2 (K-composition (weight + path-patch, GPT-2))
• discovered_write_hub_edges — GPT-2 (DISCOVERED (novel-live composition edges))
• v_virtual_heads — GPT-2 (V-composition (composed-OV ‘virtual heads’, GPT-2))

Circuit vs operator — a naming note. A few names (duplicate, induction) appear in both this circuit catalog and the operator catalog. A circuit is a composition (circuit:induction = prev-token → induction); the same-named operator is the head class it is named after and built around (op:induction). The coincidence is deliberate: a circuit is keyed by its reader operator. Pages cross-link to their namesake.

Cross-model circuit-edge liveness (remove the writer from the reader’s key → attention collapse %)

circuit	defining edge	gpt2	gpt2-medium	gpt2-large	gemma-2-2b	Llama-3.2-1B	Qwen2.5-1.5B
induction	prevtok_head -> induction (K)	+17% 4.11	+23% 2.15	+8% 3.14	+18% 0.0	+70% 1.20	+89% 1.4
positional_broadcast	sink-writer -> prevtok key (K)	+22% 1.3	+32% 1.5	+0% 3.0	+0% 5.4	(skip)	+0% 0.0
duplicate	(reader-side; writer often layer-0)	(skip)	+0% 0.13	+4% 1.1	+3% 0.2	+9% 0.17	+13% 0.6

Reading it: the induction edge (prev-token → induction) is live in every model (and stronger in RoPE — content matching lives in the key everywhere); positional-broadcast (sink/hub → prev-token key) is GPT-2-small/medium-only (the absolute-position plumbing — RoPE reads position from the rotation, so the prev-token key has no upstream writer to remove). Same absolute-position-family split as the operator catalog’s sink.

Discovered edges (de novo, cross-model)

Beyond the named circuits, discovered circuit edges runs the key-patch over the top content readers in every model and keeps the edges that collapse the reader beyond a reader-matched null. It recovers the prev-token→induction K-chain de novo in the GPT-2 family (6/2/2 live edges) and finds localized edges in Llama (3) and Qwen (1), but none in Gemma (0 — its content-reader keys aren’t sharply localized to one writer; RoPE distributes the circuit). 14 live edges total.

Executable decompilation — is the circuit sufficient?

Edge liveness shows the circuit’s edges are necessary. Reconstruction tests sufficiency: keep only the induction circuit’s heads (induction + prev-token), mean-ablate every other attention head (MLPs intact), and measure how much induction the circuit alone recovers — far above a random same-size head-set. A small head-set that reconstructs most of the behaviour is an executable decompilation.

Each cross-model circuit page now also carries a cross-model causal dossier (necessity + sufficiency + redundancy, operator-parity), generated on the unified ResidualVM debugger (circuit_dossier_xmodel.py). The sharpest read: the induction circuit’s necessity and sufficiency both decay monotonically across the GPT-2 ladder (small → XL) — the named circuit is most localized in the smallest model and dissolves into the network with scale, the same distributedness theme measured as a clean ablation battery.

MLP nodes in the circuit DAG — the COMPUTE class, cross-model

The cross-model edges above are attention-only (head→head). But circuits also route through MLPs: mlp_circuit_xmodel.py (on the ResidualVM) makes them first-class circuit nodes — per-layer COMPUTE importance for induction (ablate each MLP → Δinduction-NLL) + the head↔MLP composition edges that wire them in.

model	induction head	all-MLP-ablated Δind-NLL	dominant induction-MLP(s)	detokenizer = MLP0?
gpt2	5.1	+8.7	L0 (+8.2), L1 (+6.3)	✓
gpt2-medium	11.1	+8.9	L0 (+15.4)	✓
gpt2-large	16.0	+13.0	L0 (+10.3)	✓
gemma-2-2b	6.3	+17.5	L0 (+3.7), L17 (+1.6), L5 (+1.4)	✓
Llama-3.2-1B	10.23	+15.1	L1 (+12.8), L0 (+12.5), L15 (+1.9)	✗ (L1)
Qwen2.5-1.5B	14.3	+15.9	L2 (+14.0), L1 (+13.4), L0 (+7.3)	✗ (L2)

• The COMPUTE class is load-bearing for induction in every model — ablating all MLPs (attention intact) costs +8.7 to +17.5 induction-NLL, so a faithful induction circuit is not attention-only; the MLP nodes belong in the DAG.
• The load-bearing MLPs are early everywhere — the detokenizer / extended-embedding substrate (MLP test) — but their concentration tracks the family: GPT-2 (and Gemma) pin it to a single MLP0, while the RoPE models Llama (L1+L0) and Qwen (L2+L1+L0) spread the substrate across the first two–three MLPs (so detokenizer = MLP0 is GPT-2/Gemma-only; the embedding is assembled across early layers in RoPE). Same localized-in-GPT-2 / distributed-in-RoPE split the attention side shows. (Data: mlp_circuit_xmodel_summary.json.)

Taxonomy & gaps

• Levels: circuit (a DAG of operator nodes) → edge (writer-op → reader-op via a K/Q/V port) → the operator classes at each node (../operators/). Edges are the primitive the discovery gate scores.
• succession / greater-than — MLP-dominated; no clean attention-composition circuit (carried by the copy ops).
• SSM (Mamba) — no heads, so no composition edges; induction is present only behaviourally (ssm_induction.py).
• IOI is now cross-model (the ioi_q_chain page): the circuit’s operators (name-movers, negative/copy-suppression movers, duplicate-token initiator) and its load-bearing necessity are found behaviourally in all 6 models via the ResidualVM — closing the old “no head-set off GPT-2” gap. The precise Q-composition edge wiring stays GPT-2-validated.
• V-composition is now cross-model too (the v_virtual_heads page): the composed-OV virtual heads (an induction head’s output re-read as a downstream value) are weight-legible in all 6 models — so the full Elhage K / Q / V composition-edge triad is now measured across models (induction K-chain, IOI Q-chain, V-virtual-heads). Dynamic ΔV-out confirmation stays GPT-2-validated.
• Still GPT-2-only: full per-edge path-patch of all 22 discovered write-hub edges on the RoPE models (they are GPT-2 absolute-position plumbing — predicted absent in RoPE). The cross-model catalog covers the universal-reader edges, the IOI operators, and all three composition-edge types.

How this was made

circuit_atlas.py (cross-model edges + harvest) → circuit_catalog_doc.py (these docs). Discovery/validation: composition_dag.py, validate_new_edges.py, vcomposition.py, ioi_causal.py, self_repair.py, rung3_induction_chain.py. See ../DECOMPILATION.md.