econ-sae lifecycle walkthrough

econ-sae is a benchmark fixture for evaluating techniques that extract structured representations from sparse autoencoders, sibling to sm-sae. A stock-flow-consistent simulated macroeconomy (households, firms, banks, government) gives us exact ground truth at multiple granularities (sector, cohort, transaction kind, conjunctive composition, regime / phase). Any candidate SAE technique can be scored quantitatively against the 51-feature vocabulary across four difficulty tiers.

The headline finding of the project: different feature tiers need different (substrate, decoder, input-encoding) recipes. No single SAE recovers everything; the multi-recipe scoreboard below documents what works for each tier.

(a) The substrate: agents as vectors in R^23

Each economic agent (household, firm, government, bank) is a point in R23. The coordinates split into a conserved block (indices 0–13: stocks and per-period flows that obey exact accounting identities at every transaction) and a side block (indices 14–22: agent parameters that drive dynamics).

Coordinate layout

idxcoordblock
0moneyconserved (stock)
1inv_foodconserved (stock)
2inv_servicesconserved (stock)
3inv_durablesconserved (stock)
4debt_liabconserved (stock)
5debt_assetconserved (stock)
6labor_supplyconserved (flow)
7labor_demandconserved (flow)
8goods_out_foodconserved (flow)
9goods_out_servicesconserved (flow)
10goods_out_durablesconserved (flow)
11goods_in_foodconserved (flow)
12goods_in_servicesconserved (flow)
13goods_in_durablesconserved (flow)
14productivityside
15wageside
16priceside
17mpcside
18tax_rateside
19sectorside
20subsectorside
21expectationside
22credit_limitside

Three example agent rows (default economy, seed=0)

Column headers truncated to 6 chars; hover for full name. Cells shaded by block (conserved — blue; side — gray).

moneyinv_foinv_seinv_dudebt_ldebt_alabor_labor_goods_goods_goods_goods_goods_goods_producwagepricempctax_rasectorsubsecexpectcredit
household (young #0)0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.86 0.90 1.00 0.91 0.18 0.00 1.00 0.78 0.00
firm (food #0)1.00 80.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2.03 0.90 1.00 0.00 0.25 1.00 1.00 80.00 4.00
bank50.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.00 0.00 3.00 0.00 0.00 0.00

Household cohorts

Three cohorts split the 12 households into behavioral classes. Cohort drives MPC, productivity, reservation wage, sector preferences (CES weights over food / services / durables), and credit access (max debt-to-income).

cohortmpcproductivityres. wagetax_rateCES (food/svc/dur)max DTI
young0.920.850.900.180.25 / 0.45 / 0.301.50
prime0.781.101.100.250.30 / 0.40 / 0.300.80
retiree0.850.400.700.150.45 / 0.45 / 0.100.00

Firm sectors

Three firm sectors with distinct production technology, pricing flexibility, and cyclicality.

sectorproductivitymarkupprice stickinessinit invinit pricecyclicality
food2.501.100.30801.000.30
services1.201.250.70201.500.70
durables1.801.350.50402.001.30

Input-output network (Phase 3.4)

With io_network=True, each firm purchases intermediate goods from each sector before producing. IO_MATRIX[row][col] entries are the units of col's good needed per unit of row's output. A productivity shock to durables (the row that supplies all three columns) propagates through goods flows to every other sector.

I-O matrix: column-supplier, row-consumer.

Nine transaction kinds (vertices)

Every transaction is a double-entry vertex that updates two agents' vectors with opposite signs on the conserved block. The kinds:

wage purchase b2b_purchase tax transfer loan_origination loan_repayment interest deposit_interest dividend

All nine fire under stochastic shocks; b2b_purchase is conditional on the I-O network being enabled.

Conservation laws (the ground-truth axes)

For every period t and every trajectory:

These six identities form a constructive ground truth: every trained model can be graded against them without post-hoc interpretation. The simulator preserves them to ~1e-13 (machine precision) across every trajectory in every experiment.

(b) Running the simulator: one trajectory of 40 periods

This is the baseline (no shocks, no Phase 3 features) trajectory of the default 12 HH + 3 firm + 1 gov + 1 bank economy. The simulator runs each period through a deterministic sequence of transaction steps (interest payments, production, wages, taxes, transfers, consumption, repayment, dividends, sticky-price update, expectations update). Every flow is a double-entry transaction; conservation holds at machine precision.

Macros over time. GDP and consumption settle into a stable steady state under the balanced-budget fiscal rule. interest_rate is constant (no monetary shocks); debt grows monotonically as young HHs draw on consumer credit.

Transaction-kind occurrence

Counts per transaction kind across the trajectory. wage / purchase dominate (every period for every employed household); loan_origination is rarer (only when an agent is liquidity-constrained).

Conservation residuals (should all be ~1e-13)

identitymax residual
money_drift1.42e-13
debt_net7.11e-15
labor_balance1.78e-15
goods_balance_food1.78e-15
goods_balance_services8.88e-16
goods_balance_durables8.88e-16

These six identities are the constructive ground truth for the SAE benchmark. The simulator preserves them exactly under every combination of shocks and Phase 3 features.

(c) SAE: per-tier alignment and the multi-decoder recipe

The ground-truth feature vocabulary spans four tiers of difficulty (categorical, bucketed, conjunctive, regime). Each tier has a different recovery recipe; no single (substrate, SAE) pair recovers everything.

Experiments aggregated below:

Per-feature AUC over the project's six experimental phases. The red dashed line is the AUC = 0.95 cov95 threshold. Some features were already recoverable at Phase 1 (the impulse features moved later when impulse flags were added to the input). Others (expansion / contraction) needed the supervised regime head in Phase 5.1 to climb above 0.90.

Best-in-class per-tier recovery

categorical — 20/31 at AUC≥0.95, mean = 0.865

showing best 6 + worst 6 of 31 categorical features.

featurebest AUCrecipe
cohort:retiree1.000Phase 1.7 temporal
firm_sector:durables1.000Phase 1.7 temporal
firm_sector:food1.000Phase 1.7 temporal
firm_sector:services1.000Phase 1.7 temporal
sector:bank1.000Phase 1.7 temporal
sector:firm1.000Phase 1.7 temporal
shock_period_has:tfp_durables0.533Phase 4.2 full features
shock_period_has:sentiment_young0.528Phase 4.2 full features
shock_period_has:sentiment_prime0.526Phase 4.2 full features
shock_period_has:tfp_services0.522Phase 4.2 full features
shock_period_has:sentiment_retiree0.522Phase 4.2 full features
shock_period_has:tfp_food0.521Phase 4.2 full features

bucketed — 3/7 at AUC≥0.95, mean = 0.902

featurebest AUCrecipe
debt_bucket:high1.000Phase 4.2 full features
inventory_bucket:stockout0.979Phase 1.7 temporal
leverage:high0.961Phase 1.7 temporal
productivity:high0.871Phase 4.2 full features
cash_bucket:rich0.866Phase 1.7 temporal
mpc_bucket:high0.820Phase 1.7 temporal
cash_bucket:poor0.815Phase 4.2 full features

conjunctive — 6/8 at AUC≥0.95, mean = 0.980

featurebest AUCrecipe
food_firm_low_inv1.000Phase 1.5 scale
services_firm_high_output1.000Phase 1.7 temporal
durables_firm_high_inv1.000Phase 1.6 attention
prime_AND_high_cash0.989Phase 1.6 attention
young_AND_high_mpc_AND_expansion0.987Phase 1.7 temporal
firm_AND_indebted_AND_high_inventory0.980Phase 4.2 full features
retiree_AND_decumulating0.944Phase 1.7 temporal
young_AND_indebted0.940Phase 1.6 attention

regime — 4/6 at AUC≥0.95, mean = 0.976

featurebest AUCrecipe
phase:fiscal_active1.000Phase 2.0 macro-feed v2
phase:high_rate1.000Phase 5.1 regime-sup
phase:monetary_active1.000Phase 4.2 full features
phase:high_leverage0.995Phase 5.1 regime-sup
phase:expansion0.940Phase 5.1 regime-sup
phase:contraction0.918Phase 5.1 regime-sup

(d) Polygram bridge

The polygram bridge encodes econ-sae's full 51-feature vocabulary as a polygram Dictionary using HEA_Rung2 with n_qubits = 6 (64 feature slots, no truncation). Per-feature beta = best-recovered AUC minus 0.5, so the "interpretability strength" of each feature appears as the scalar on the Dictionary slot.

Encoding-capacity diagnostics

Compression-style diagnostics mirroring sae-forge's ParetoFrontierRow.polygram_* fields. Headroom = capacity − n_features; positive means under-utilised substrate. utilisation = how much of the encoding's 2n_qubits slot space is occupied by real features.

n_featuresencodingn_qubits capacity (= 2n_qubits) headroomutilisationn_clusters
52 HEA_Rung2 6 64 +12 81.2% 52

Per-tier beta spread

tiernbeta minbeta medianbeta max
bucketed7+0.305+0.371+0.498
categorical31+0.000+0.073+0.500
conjunctive8+0.424+0.488+0.500
regime6+0.282+0.384+0.500

Interference sweep: rotate phase on a cross-tier pair

Target pair firm_sector_food / food_firm_low_inv, sweeping firm_sector_food.phi from 0 to 2π over 60 values.

Target-pair overlap range: 0.7704 to 1.0000 (mean 0.8870).

Cancellation experiments

For each pair, polygram searches for phase values that drive the inner product to zero. The "after" column is what phase tuning alone achieves under the HEA_Rung2 encoding's structural floor (~0.77 here, similar across pairs). Cross-tier and within-tier pairs converge to similar floors, which is informative about the encoding's geometry.

labelpairoverlap beforeafterefficiencymet?
cat_cat_distantsector_bank / firm_sector_food1.00000.7705N/AFalse
cat_cat_same_axisfirm_sector_food / firm_sector_durables1.00000.7705N/AFalse
buck_buckdebt_high / mpc_high0.99210.8421N/AFalse
conj_conjfood_firm_low_inv / young_AND_indebted0.99910.7952N/AFalse
cat_in_conjfirm_sector_food / food_firm_low_inv1.00000.7706N/AFalse
cat_vs_regimesector_household / phase_high_leverage0.99980.7815N/AFalse
regime_regimephase_high_leverage / phase_fiscal_active0.99980.7812N/AFalse
regime_vs_conjphase_high_leverage / young_AND_indebted0.99970.8054N/AFalse
cohort_cohortcohort_young / cohort_retiree1.00000.7706N/AFalse

(e) Final benchmark scoreboard

Best-in-class per-tier results across all five experimental phases. The recipe column lists the (substrate, decoder) pair that achieved each tier's best mAUC: different tiers need different recipes. No single SAE / world-model combination recovers every tier; the multi-decoder recipe is the project's headline finding.

tiernbest mAUCbest feature recoveriesrecipe
categorical301.000most at 1.000per-agent SAE on attention substrate
bucketed70.928most 0.85–0.93per-agent SAE
conjunctive80.9686/8 at AUC≥0.95 (with I-O network + attention)per-agent SAE on Phase-3 substrate
regime60.9724/6 at AUC≥0.95 (high_leverage, fiscal, monetary, high_rate)per-period macro-feed v3 SAE on regime-supervised WM
windowed regime(subset of regime)0.920remaining 2 features (expansion, contraction) at AUC 0.92needs feature-bottlenecked supervision or specialized SAE

The recommended two-decoder recipe

  1. Train an attention-enabled world model (cross-agent attention block + per-agent MLP head, optionally with cross-period GRU) on the multi-trajectory ensemble.
  2. Train a per-agent SAE on its h1 activations to recover the categorical, bucketed, and conjunctive tiers.
  3. Train a per-period macro-feed SAE on [macro + shock + GDP_window + impulse_flags + engineered ratios + mean-pooled h1] (z-scored) to recover the regime tier.
  4. Optionally add a supervised regime head during world-model training to push the windowed-regime sub-features (expansion / contraction) toward AUC 0.95.

(f) Phase journey: how the ceiling moved at each step

Five lessons compound across the journey:

  1. Substrate architecture sets the ceiling for per-agent features. Attention block was the single biggest unlock for conjunctive recovery.
  2. Decoder granularity matters as much as substrate quality. Per-agent SAE for categorical / bucketed / conjunctive; per-period SAE for regime.
  3. Input encoding matters as much as decoder choice. Impulse flags and engineered ratios each closed a different sub-tier of regime features.
  4. Simulator dynamics drive what's recoverable. I-O cascades made one conjunctive feature cleanly recoverable for the first time.
  5. Some labels need supervision. Threshold-on-ratio regime labels stalled at AUC 0.87 with unsupervised SAEs; supervised regime heads pushed them to 1.00 (for high_rate) and 0.92 (for expansion / contraction).
Phase 0.5Multi-good, multi-cohort, credit-enabled simulator. 9 transaction kinds, 6 conservation identities preserved at ~1e-13. Stochastic shock generator + ensemble runner. 51-feature ground-truth vocabulary across 4 difficulty tiers.
Phase 1Per-agent SAE pipeline: TopK / L1 / JumpReLU on three feeds (raw, embedded, world-model activations). Categorical features recover trivially. Conjunctive mAUC 0.84, regime mAUC 0.60.
Phase 1.5Width sweep (256 to 1024). Conjunctive cov95 climbs from 4/8 to 5/8. Width helps modestly; duration alone is a no-op.
Phase 1.6AttnWorldModel: cross-agent attention block. Conjunctive mAUC 0.84 -> 0.97; 6/8 features at AUC≥0.95. Biggest single architectural unlock for the per-agent tiers.
Phase 1.7TemporalWorldModel (attention + GRU across periods). Falsified the 'temporal architecture unlocks regime' hypothesis. Diagnosis: next-state MSE doesn't reward history encoding.
Phase 1.8Sentiment-driven MPC in the simulator. Falsified the 'Markovian simulator' diagnosis. Agent state already encodes history through accumulated stocks.
Phase 1.9Macro-feed SAE: per-period samples. phase:high_leverage jumps from 0.62 to 0.97. Confirms the structural diagnosis: regime is per-period, not per-agent.
Phase 2.0Macro-feed v2: GDP window + impulse flags + z-scored input. fiscal_active and monetary_active cross AUC 0.99. Regime mAUC 0.86.
Phase 3Polygram bridge (full 51-feature dictionary, HEA_Rung2). GatedSAE (neutral on plateaued features). Taylor rule. Input-output firm network.
Phase 4.1Macro-feed v3: engineered ratio inputs (gdp_deviation, rate_deviation, leverage_ratio, inflation). phase:high_leverage back to 0.97. Regime mAUC 0.885.
Phase 4.2Full Phase 3 features (sentiment + Taylor + I-O) in an SAE pipeline. firm_AND_indebted_AND_high_inventory crosses AUC 0.95 for the first time (0.92 -> 0.98) thanks to I-O cascades.
Phase 5.1Regime-supervised TemporalWorldModel. Regime mAUC 0.885 -> 0.972 (biggest single-phase jump). 4/6 regime features cleanly recovered. The remaining 2 (expansion, contraction) climb to 0.92 but plateau due to distributed-encoding limitation of pooled supervision.