econ-sae is a benchmark fixture for evaluating techniques that extract structured representations from sparse autoencoders, sibling to sm-sae. A stock-flow-consistent simulated macroeconomy (households, firms, banks, government) gives us exact ground truth at multiple granularities (sector, cohort, transaction kind, conjunctive composition, regime / phase). Any candidate SAE technique can be scored quantitatively against the 51-feature vocabulary across four difficulty tiers.
The headline finding of the project: different feature tiers need different (substrate, decoder, input-encoding) recipes. No single SAE recovers everything; the multi-recipe scoreboard below documents what works for each tier.
Each economic agent (household, firm, government, bank) is a point in R23. The coordinates split into a conserved block (indices 0–13: stocks and per-period flows that obey exact accounting identities at every transaction) and a side block (indices 14–22: agent parameters that drive dynamics).
| idx | coord | block |
|---|---|---|
| 0 | money | conserved (stock) |
| 1 | inv_food | conserved (stock) |
| 2 | inv_services | conserved (stock) |
| 3 | inv_durables | conserved (stock) |
| 4 | debt_liab | conserved (stock) |
| 5 | debt_asset | conserved (stock) |
| 6 | labor_supply | conserved (flow) |
| 7 | labor_demand | conserved (flow) |
| 8 | goods_out_food | conserved (flow) |
| 9 | goods_out_services | conserved (flow) |
| 10 | goods_out_durables | conserved (flow) |
| 11 | goods_in_food | conserved (flow) |
| 12 | goods_in_services | conserved (flow) |
| 13 | goods_in_durables | conserved (flow) |
| 14 | productivity | side |
| 15 | wage | side |
| 16 | price | side |
| 17 | mpc | side |
| 18 | tax_rate | side |
| 19 | sector | side |
| 20 | subsector | side |
| 21 | expectation | side |
| 22 | credit_limit | side |
Column headers truncated to 6 chars; hover for full name. Cells shaded by block (conserved — blue; side — gray).
| money | inv_fo | inv_se | inv_du | debt_l | debt_a | labor_ | labor_ | goods_ | goods_ | goods_ | goods_ | goods_ | goods_ | produc | wage | price | mpc | tax_ra | sector | subsec | expect | credit | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| household (young #0) | 0.05 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.86 | 0.90 | 1.00 | 0.91 | 0.18 | 0.00 | 1.00 | 0.78 | 0.00 |
| firm (food #0) | 1.00 | 80.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2.03 | 0.90 | 1.00 | 0.00 | 0.25 | 1.00 | 1.00 | 80.00 | 4.00 |
| bank | 50.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.02 | 0.00 | 0.00 | 3.00 | 0.00 | 0.00 | 0.00 |
Three cohorts split the 12 households into behavioral classes. Cohort drives MPC, productivity, reservation wage, sector preferences (CES weights over food / services / durables), and credit access (max debt-to-income).
| cohort | mpc | productivity | res. wage | tax_rate | CES (food/svc/dur) | max DTI |
|---|---|---|---|---|---|---|
young | 0.92 | 0.85 | 0.90 | 0.18 | 0.25 / 0.45 / 0.30 | 1.50 |
prime | 0.78 | 1.10 | 1.10 | 0.25 | 0.30 / 0.40 / 0.30 | 0.80 |
retiree | 0.85 | 0.40 | 0.70 | 0.15 | 0.45 / 0.45 / 0.10 | 0.00 |
Three firm sectors with distinct production technology, pricing flexibility, and cyclicality.
| sector | productivity | markup | price stickiness | init inv | init price | cyclicality |
|---|---|---|---|---|---|---|
food | 2.50 | 1.10 | 0.30 | 80 | 1.00 | 0.30 |
services | 1.20 | 1.25 | 0.70 | 20 | 1.50 | 0.70 |
durables | 1.80 | 1.35 | 0.50 | 40 | 2.00 | 1.30 |
With io_network=True, each firm purchases intermediate
goods from each sector before producing. IO_MATRIX[row][col] entries are
the units of col's good needed per unit of row's output.
A productivity shock to durables (the row that supplies all
three columns) propagates through goods flows to every other sector.
Every transaction is a double-entry vertex that updates two agents' vectors with opposite signs on the conserved block. The kinds:
wage purchase b2b_purchase tax transfer loan_origination loan_repayment interest deposit_interest dividend
All nine fire under stochastic shocks; b2b_purchase
is conditional on the I-O network being enabled.
For every period t and every trajectory:
sum_i money[i] = M_initial (closed economy)sum_i (debt_liab - debt_asset)[i] = 0 (every loan two-sided)sum_i labor_supply[i] = sum_i labor_demand[i]sum_i goods_in_food[i] = sum_i goods_out_food[i]
(and services, durables)These six identities form a constructive ground truth: every trained model can be graded against them without post-hoc interpretation. The simulator preserves them to ~1e-13 (machine precision) across every trajectory in every experiment.
This is the baseline (no shocks, no Phase 3 features) trajectory of the default 12 HH + 3 firm + 1 gov + 1 bank economy. The simulator runs each period through a deterministic sequence of transaction steps (interest payments, production, wages, taxes, transfers, consumption, repayment, dividends, sticky-price update, expectations update). Every flow is a double-entry transaction; conservation holds at machine precision.
| identity | max residual |
|---|---|
money_drift | 1.42e-13 |
debt_net | 7.11e-15 |
labor_balance | 1.78e-15 |
goods_balance_food | 1.78e-15 |
goods_balance_services | 8.88e-16 |
goods_balance_durables | 8.88e-16 |
These six identities are the constructive ground truth for the SAE benchmark. The simulator preserves them exactly under every combination of shocks and Phase 3 features.
The ground-truth feature vocabulary spans four tiers of difficulty (categorical, bucketed, conjunctive, regime). Each tier has a different recovery recipe; no single (substrate, SAE) pair recovers everything.
Experiments aggregated below:
showing best 6 + worst 6 of 31 categorical features.
| feature | best AUC | recipe |
|---|---|---|
cohort:retiree | 1.000 | Phase 1.7 temporal |
firm_sector:durables | 1.000 | Phase 1.7 temporal |
firm_sector:food | 1.000 | Phase 1.7 temporal |
firm_sector:services | 1.000 | Phase 1.7 temporal |
sector:bank | 1.000 | Phase 1.7 temporal |
sector:firm | 1.000 | Phase 1.7 temporal |
shock_period_has:tfp_durables | 0.533 | Phase 4.2 full features |
shock_period_has:sentiment_young | 0.528 | Phase 4.2 full features |
shock_period_has:sentiment_prime | 0.526 | Phase 4.2 full features |
shock_period_has:tfp_services | 0.522 | Phase 4.2 full features |
shock_period_has:sentiment_retiree | 0.522 | Phase 4.2 full features |
shock_period_has:tfp_food | 0.521 | Phase 4.2 full features |
| feature | best AUC | recipe |
|---|---|---|
debt_bucket:high | 1.000 | Phase 4.2 full features |
inventory_bucket:stockout | 0.979 | Phase 1.7 temporal |
leverage:high | 0.961 | Phase 1.7 temporal |
productivity:high | 0.871 | Phase 4.2 full features |
cash_bucket:rich | 0.866 | Phase 1.7 temporal |
mpc_bucket:high | 0.820 | Phase 1.7 temporal |
cash_bucket:poor | 0.815 | Phase 4.2 full features |
| feature | best AUC | recipe |
|---|---|---|
food_firm_low_inv | 1.000 | Phase 1.5 scale |
services_firm_high_output | 1.000 | Phase 1.7 temporal |
durables_firm_high_inv | 1.000 | Phase 1.6 attention |
prime_AND_high_cash | 0.989 | Phase 1.6 attention |
young_AND_high_mpc_AND_expansion | 0.987 | Phase 1.7 temporal |
firm_AND_indebted_AND_high_inventory | 0.980 | Phase 4.2 full features |
retiree_AND_decumulating | 0.944 | Phase 1.7 temporal |
young_AND_indebted | 0.940 | Phase 1.6 attention |
| feature | best AUC | recipe |
|---|---|---|
phase:fiscal_active | 1.000 | Phase 2.0 macro-feed v2 |
phase:high_rate | 1.000 | Phase 5.1 regime-sup |
phase:monetary_active | 1.000 | Phase 4.2 full features |
phase:high_leverage | 0.995 | Phase 5.1 regime-sup |
phase:expansion | 0.940 | Phase 5.1 regime-sup |
phase:contraction | 0.918 | Phase 5.1 regime-sup |
The polygram bridge encodes econ-sae's full 51-feature vocabulary as a
polygram Dictionary using HEA_Rung2 with n_qubits =
6 (64 feature slots,
no truncation). Per-feature beta = best-recovered AUC minus 0.5,
so the "interpretability strength" of each feature appears as the scalar
on the Dictionary slot.
Compression-style diagnostics mirroring sae-forge's
ParetoFrontierRow.polygram_* fields. Headroom = capacity −
n_features; positive means under-utilised substrate. utilisation = how
much of the encoding's 2n_qubits slot space is occupied by
real features.
| n_features | encoding | n_qubits | capacity (= 2n_qubits) | headroom | utilisation | n_clusters |
|---|---|---|---|---|---|---|
| 52 | HEA_Rung2 |
6 | 64 | +12 | 81.2% | 52 |
| tier | n | beta min | beta median | beta max |
|---|---|---|---|---|
bucketed | 7 | +0.305 | +0.371 | +0.498 |
categorical | 31 | +0.000 | +0.073 | +0.500 |
conjunctive | 8 | +0.424 | +0.488 | +0.500 |
regime | 6 | +0.282 | +0.384 | +0.500 |
Target pair firm_sector_food /
food_firm_low_inv, sweeping
firm_sector_food.phi from 0 to 2π over
60 values.
Target-pair overlap range: 0.7704 to 1.0000 (mean 0.8870).
For each pair, polygram searches for phase values that drive the inner product to zero. The "after" column is what phase tuning alone achieves under the HEA_Rung2 encoding's structural floor (~0.77 here, similar across pairs). Cross-tier and within-tier pairs converge to similar floors, which is informative about the encoding's geometry.
| label | pair | overlap before | after | efficiency | met? |
|---|---|---|---|---|---|
cat_cat_distant | sector_bank / firm_sector_food | 1.0000 | 0.7705 | N/A | False |
cat_cat_same_axis | firm_sector_food / firm_sector_durables | 1.0000 | 0.7705 | N/A | False |
buck_buck | debt_high / mpc_high | 0.9921 | 0.8421 | N/A | False |
conj_conj | food_firm_low_inv / young_AND_indebted | 0.9991 | 0.7952 | N/A | False |
cat_in_conj | firm_sector_food / food_firm_low_inv | 1.0000 | 0.7706 | N/A | False |
cat_vs_regime | sector_household / phase_high_leverage | 0.9998 | 0.7815 | N/A | False |
regime_regime | phase_high_leverage / phase_fiscal_active | 0.9998 | 0.7812 | N/A | False |
regime_vs_conj | phase_high_leverage / young_AND_indebted | 0.9997 | 0.8054 | N/A | False |
cohort_cohort | cohort_young / cohort_retiree | 1.0000 | 0.7706 | N/A | False |
Best-in-class per-tier results across all five experimental phases. The recipe column lists the (substrate, decoder) pair that achieved each tier's best mAUC: different tiers need different recipes. No single SAE / world-model combination recovers every tier; the multi-decoder recipe is the project's headline finding.
| tier | n | best mAUC | best feature recoveries | recipe |
|---|---|---|---|---|
| categorical | 30 | 1.000 | most at 1.000 | per-agent SAE on attention substrate |
| bucketed | 7 | 0.928 | most 0.85–0.93 | per-agent SAE |
| conjunctive | 8 | 0.968 | 6/8 at AUC≥0.95 (with I-O network + attention) | per-agent SAE on Phase-3 substrate |
| regime | 6 | 0.972 | 4/6 at AUC≥0.95 (high_leverage, fiscal, monetary, high_rate) | per-period macro-feed v3 SAE on regime-supervised WM |
| windowed regime | (subset of regime) | 0.920 | remaining 2 features (expansion, contraction) at AUC 0.92 | needs feature-bottlenecked supervision or specialized SAE |
[macro + shock + GDP_window + impulse_flags + engineered ratios
+ mean-pooled h1] (z-scored) to recover the regime tier.Five lessons compound across the journey:
| Phase 0.5 | Multi-good, multi-cohort, credit-enabled simulator. 9 transaction kinds, 6 conservation identities preserved at ~1e-13. Stochastic shock generator + ensemble runner. 51-feature ground-truth vocabulary across 4 difficulty tiers. |
|---|---|
| Phase 1 | Per-agent SAE pipeline: TopK / L1 / JumpReLU on three feeds (raw, embedded, world-model activations). Categorical features recover trivially. Conjunctive mAUC 0.84, regime mAUC 0.60. |
| Phase 1.5 | Width sweep (256 to 1024). Conjunctive cov95 climbs from 4/8 to 5/8. Width helps modestly; duration alone is a no-op. |
| Phase 1.6 | AttnWorldModel: cross-agent attention block. Conjunctive mAUC 0.84 -> 0.97; 6/8 features at AUC≥0.95. Biggest single architectural unlock for the per-agent tiers. |
| Phase 1.7 | TemporalWorldModel (attention + GRU across periods). Falsified the 'temporal architecture unlocks regime' hypothesis. Diagnosis: next-state MSE doesn't reward history encoding. |
| Phase 1.8 | Sentiment-driven MPC in the simulator. Falsified the 'Markovian simulator' diagnosis. Agent state already encodes history through accumulated stocks. |
| Phase 1.9 | Macro-feed SAE: per-period samples. phase:high_leverage jumps from 0.62 to 0.97. Confirms the structural diagnosis: regime is per-period, not per-agent. |
| Phase 2.0 | Macro-feed v2: GDP window + impulse flags + z-scored input. fiscal_active and monetary_active cross AUC 0.99. Regime mAUC 0.86. |
| Phase 3 | Polygram bridge (full 51-feature dictionary, HEA_Rung2). GatedSAE (neutral on plateaued features). Taylor rule. Input-output firm network. |
| Phase 4.1 | Macro-feed v3: engineered ratio inputs (gdp_deviation, rate_deviation, leverage_ratio, inflation). phase:high_leverage back to 0.97. Regime mAUC 0.885. |
| Phase 4.2 | Full Phase 3 features (sentiment + Taylor + I-O) in an SAE pipeline. firm_AND_indebted_AND_high_inventory crosses AUC 0.95 for the first time (0.92 -> 0.98) thanks to I-O cascades. |
| Phase 5.1 | Regime-supervised TemporalWorldModel. Regime mAUC 0.885 -> 0.972 (biggest single-phase jump). 4/6 regime features cleanly recovered. The remaining 2 (expansion, contraction) climb to 0.92 but plateau due to distributed-encoding limitation of pooled supervision. |