Binding transformer

Study 001 / seed 42 / live model

Held-out accuracy—

Answer loss nats—

Training step0

Observed units—/ 256

Measuring

Measuring the model

Building fingerprints from the fixed calibration prompts.

Layer 1 Layer 2Preparing atlasNeighbors retained —Variance —

PROBE 01/16

t = 13

CHECKPOINT

LIVE

Analysis & evidence Activations, learning curves and repair experiments

Checkpoint — · calibration only

Activation stripsPOST-ReLU · TOKEN × CHANNEL

024+

Activation measurements appear after a probe is captured.

Answer lossUniform answer prior

96 fixed evaluation prompts
Chance 12.5% · input-copy baseline 33.3%

Model & runs

Model anatomy

25.9K

25.9k weights

Embeddingstoken + position · 14 × 32 eachRMS norm → 32d residual

ATTNRMS → causal attention4 heads × 8d → Wᵒ+ x

MLPRMS → 32 → 128 → 32ReLU · 128 atlas units+ x′

Nodes 0–127

ATTNRMS → causal attention4 heads × 8d → Wᵒ+ x

MLPRMS → 32 → 128 → 32ReLU · 128 atlas units+ x′

Nodes 128–255

Last-token readoutlmHead [32 × 14] → softmax

256 post-ReLU MLP points, reused per token. Attention is not plotted.

2 blocks4 heads32 width

New seed

Training settings

Updates