— P R I S M timepointai / nanogpt-prism-shakespeare · v0.2 · repo · whitepaper
A spectrographic flashlight for data

Point it at a trained model and it reads out — as a spectrum — how the model organized itself to learn.
Then run the prism backwards into a fresh one.

Every trained network produces two things: the weights (what it learned) and a spectral blueprint (how it arranged itself to learn it). Standard practice throws the blueprint away. Prism extracts it — the singular-value spectrum and singular directions of every weight matrix — and uses it to initialize and continuously regularize a new model. No parameters are copied. Only geometry.

The original idea, kept as the working metaphor. A spectrographic flashlight for data: shine it at information — here, a trained checkpoint — and it parses the light into attribute bands you can measure, store, and re-project. The left prism is extraction (SVD per weight group); the right prism runs backwards (initialization + regularization of a fresh model). What travels between them is geometry, never content.

00The metaphor, made precise

Every part of the picture is a real mathematical object.

in the picturein the mathwhat it is
white lightW ∈ ℝm×n a trained weight matrix — content and organization mixed together, unreadable directly
the prismW = U Σ Vᵀ singular value decomposition — the one factorization that cleanly separates how much energy from which directions
the spectrumσ₁ ≥ σ₂ ≥ … ≥ σᵣ the singular values: how the model distributes representational energy. Trained models have heavy-tailed, decidedly non-random spectra
each color bandweight group attention, FFN-up, FFN-down, embedding — matrices of the same role share a characteristic spectral shape, so their spectra are averaged per group
the photograph
of the spectrum
8 DCT coefficients the group-averaged spectrum is a smooth decaying curve; a least-squares fit of 8 cosine terms (in inverse-softplus space) captures it to ~0.03 mean abs error. ~128 bytes total
polarization
of each band
U, V the singular directions: which subspaces the model decided were worth using. Large (~500 MB uncompressed), dimension-specific
the reversed prismimprint · align · hold the three write-side operations (§02): reshape the student's spectrum, rotate its directions toward the teacher's, and keep pulling it back as it trains
01One node — a single weight matrix

What the flashlight sees when it hits one matrix.

Take any trained weight matrix and decompose it: W = UΣVᵀ. The spectrum Σ says how much the matrix concentrates its energy into few directions (trained matrices concentrate hard; random ones don't). The directions U, V say where.

The spectrum compresses brutally well — it is a smooth curve, and eight cosine coefficients reconstruct it almost exactly. The directions do not compress; they are carried whole. That asymmetry is honest and load-bearing: "128 bytes" describes the spectral tier of the method, not all of it.

On the write side, three operations move the geometry into a fresh matrix W′ — and the third one never stops running.

Why the third operation matters most. Overfitting, seen geometrically, is the weight matrices drifting out of the task-aligned subspace they should occupy. The mod wheel makes the transferred shape a standing constraint, not just a starting point — which is why the recipe holds its best loss for thousands of steps where the from-scratch baseline peaks and collapses.

02The whole graph — a model's matrices as nodes

Geometry moves between graphs. Content never does.

Zoom out: a transformer is a graph of weight matrices. Prism treats every node the same way — but the spectrum is shared per color (weight group), while the directions stay per-node. Four spectral photographs describe the energy layout of the entire network; each node additionally receives its own directional rotation.

Look at what crosses the gap and what doesn't. The teacher's activations, logits, data, and learned associations stay home. What crosses is the organizational grammar: how sharply each kind of matrix concentrates energy, and which axes it concentrates it on.

This is what makes the measured results possible: a student trained on disjoint data — or a different corpus — inherits the full head start, because the head start was never about the data.

Reading the picture. Node color = weight group · ring thickness = spectral energy · dashed halo on the student = the mod wheel's standing pull. The four pooled spectrum chips are the "photograph" — the whole network's energy layout in ~128 bytes. Directions travel per-node (the heavy tier). Gradients on the right come from the student's own data only; the measured results below show it does not matter whether that data overlaps the teacher's at all.

03What the measurements say

Every number has a committed artifact. That is the house rule.

nanoGPT Shakespeare-char, 10.65M parameters, three seeds per run, NVIDIA L4. Artifacts live in results/ with full loss curves, provenance, and censoring flags.

11.8× median
Steps to the from-scratch baseline's best quality: ~100 vs ~1,200. Resolved, not a bound (10.2–11.9×). 7.0× under the strictest control — baseline's own LR, only spectral flags differ.
recipe_20260721T142104Z · recipe_20260720T230405Z
flat across overlap
The early advantage (~23% lower loss) is identical whether teacher and student share 100% or 0% of their data — difficulty-controlled, 12 overlap points. It was never about shared content.
recipe_20260721T050203Z
0.59→0.63 Δloss
Cross-domain: a Shakespeare teacher accelerates learning of Sherlock Holmes — student trained and scored on Sherlock — at least as much as Shakespeare. The gap grows with distance.
recipe_20260721T161208Z
saturates ≈2k lever
The advantage grows monotonically with teacher training (−0.069 → +0.46) and plateaus at ~2,000 steps — right where the teacher itself converges (4k/8k flat). Sharp edge: a weak teacher is worse than random init.
recipe_20260721T143246Z · recipe_20260721T172238Z
HONEST BOUNDS
  • The transfer measurements are early-window probes (step 100, init-dominated); the speed and no-overfitting results are longer runs (1,500–5,000 steps) but same-data.
  • "Different corpus" means Sherlock Holmes — a different author, still English prose over the same character vocabulary. Code or another language is the untested frontier.
  • Spectral vs. generic regularization is not yet isolated by a direct control; the teacher-strength dependence is suggestive, not a substitute.
  • Everything is 10.65M parameters and ~1M-token corpora. Nothing here has met a production model.
04The whole thing in one breath

A trained checkpoint carries a reusable structural prior.
Prism is the instrument that extracts and applies it.

The student never sees what the teacher knew. It inherits how the teacher was shaped — and that shape turns out to be most of what the first thousand steps of training laboriously rediscover, largely independent of the data used to discover it.

And the same instrument works the other way: kept on while you finetune a trained model — anchored to its own weights — it cuts catastrophic forgetting up to ~10×. The twist is that that protection rides the singular directions, not the spectrum — so the geometry splits in two: the spectrum is transferable structure, the directions are retained content. Structure transfers; content is retained. The other direction →

machine-readable summary (for agents reviewing this page)
method: Prism — a trained model's geometry, used two ways: (1) TRANSFER it into a
        fresh model (from-scratch spectral transfer), and (2) ANCHOR a trained model
        with it during FINETUNING (finetune-retention). Sections below.

(1) from-scratch spectral transfer learning
  extract (read):  per weight matrix W = U·Σ·Vᵀ; pool normalized singular-value
                   spectra per weight group {attention, ffn_up, ffn_down, embedding};
                   fit each pooled spectrum with 8 DCT coefficients via least squares
                   in inverse-softplus space (~128 bytes total); keep U,V per matrix.
  apply (write):   1) spectral imprint — reshape student singular values to the
                      reconstructed teacher spectrum at init;
                   2) eigentransfer — blend student singular vectors 75% toward
                      teacher's, re-orthogonalize;
                   3) mod wheel — after every optimizer step,
                      W ← (1−s)·W + s·W_target, s=0.01 decaying ×0.9999/step.
  not transferred: weights, activations, data, learned content.

measured (nanoGPT Shakespeare-char, 10.65M params, 3 seeds, committed artifacts):
  speed:        11.8× median steps-to-baseline-best (resolved); 7.0× with schedule
                matched (only spectral flags differ) — attribution control.
  overfitting:  recipe stable through 5,000 steps at both LRs tested; baseline
                peaks ~step 1,350 then decays 1.78 → ~2.31 on all seeds.
  structure:    early advantage (~23% lower loss @ step 100) flat across
                teacher/student data overlap 1.0 → 0.0 (difficulty-controlled).
  cross-domain: advantage persists (Δloss 0.591 → 0.627) when student trains AND
                is evaluated on a different corpus (Sherlock Holmes, token-JS 0.027).
  teacher lever: advantage monotonic in teacher training, saturating at ~2,000
                steps (where the teacher itself converges; 4k/8k flat at Δ≈+0.46);
                100-step teacher is WORSE than random init (Δ −0.069).
  lever evals:  plain 75% full-direction blend beats grassmann pairing (which
                eliminates the head start) and top-k=128 (uniformly ~0.05 worse)
                at Sherlock-distance.

(2) finetune-retention (the other direction; docs/FINETUNE-RETENTION.md)
  apply (write):  keep the mod wheel ON during a finetune (init_from=resume),
                  self-anchored to the model's own pre-finetune weights, constant
                  pull (prism_anchor_mode=raw, decay=1.0). Entry point:
                  prism_finetune.py; benchmark: prism_finetune_eval.py.
  measured (Shakespeare base → Sherlock finetune, 1000 steps, 3 seeds):
    retention:    up to ~10× less catastrophic forgetting — plain forgets +0.428
                  nats of the old domain, raw anchor(0.02) only +0.043; still beats
                  the from-scratch new-domain ceiling (learns, not frozen).
    attribution:  it is NOT the spectrum — anchoring spectrum-only (free directions)
                  forgets like plain (1.07×); a wrong-spectrum placebo HARMS (0.39×);
                  the raw whole-weight anchor Pareto-dominates the low-LR frontier
                  (~2× more retention at equal adaptation). Protection is a raw
                  DIRECTIONAL/weight anchor (L2-to-init / EWC-lite), not spectral.
    the split:    spectrum = transferable STRUCTURE (regime 1); directions = retained
                  CONTENT (regime 2). Structure transfers; content is retained.

(3) the arc — regimes 1 and 2 COMPOUND (docs/UNIFIED-ARC.md)
  measured (3 seeds, matched Shakespeare val ~1.79, prism-vs-plain base, same anchor):
    a PRISM-PRETRAINED base finetunes with ~ZERO forgetting (vs +0.05 for a plain base)
    and adapts ~8% better — and a schedule-matched plain control (plain at prism's LR,
    no spectral) forgets MORE, so it is the SPECTRAL GEOMETRY, not the learning rate. A
    spectrally-healthy base is a better anchor; PRISM pretraining + finetuning reinforce.

(4) prior-fusion — geometry COMPOUNDS with statistics (docs/PRIOR-FUSED-PRISM.md)
  fuse a fixed shared n-gram prior into the logits (product of experts: final =
  model_logits + lambda*log p_ngram(next|last C chars)); model learns only the residual.
  measured (1 seed, steps-to-baseline-best): PRISM alone 15x, n-gram prior alone 3.8x,
  HYBRID 30x — double PRISM — reaching the best loss of all (below baseline, prism-alone,
  and the n-gram floor; PRISM's geometry breaks the residual below the prior). A context-3
  char n-gram already predicts Shakespeare val at ~2.57 bits/char = the neural baseline.
  (literal 1000x reach-at-init is a knife-edge; needs a context-4+ prior below baseline.)

bounds: transfer results are 100-step probes; far corpus is same-language prose;
        no generic-regularization control yet; 10.65M params only. Finetune-retention
        shows old-domain RETENTION (not new-domain overfit prevention), far=Sherlock.
requires: a TRAINED model of the same architecture (dimension-specific directions).
evidence: github.com/timepointai/nanogpt-prism-shakespeare → results/*.json
          (loss curves, provenance, censoring flags; partial runs refuse to score).