J-space analysis of Inkling: one geometry across the stack

Summary

Prime Intellect replication of Anthropic Jacobian-lens / J-space geometry on open models. Inkling (thinkingmachines/Inkling-NVFP4, ~950B MoE) is an outlier vs the usual near-orthogonal sensory/workspace/motor split: early-late CKA ~0.8 vs ~0.5 elsewhere, i.e. roughly one geometry across the stack. Fit used 250 WikiText prompts and 65 source layers (vs Neuronpedia’s typical 1000). Companion A/B on Poolside Laguna XS 2.1 bf16 vs NVFP4 (200 shared prompts, bit-identical probes): quantization barely moves J-space. Interactive CKA explorer plus open-jlens-data (runs/E7_inkling, E8_laguna). Distinct from the Inkling weights themselves (papers_local 656). Underlying method paper Gurnee et al. Verbalizable Representations Form a Global Workspace (transformer-circuits.pub/2026/workspace).

Keywords

j-space · j-lens · inkling · laguna · cka · interpretability · prime-intellect · x

Topics

interpretability, J-space, MoE geometry

Research notes

• Primary: HF Jacobian-lens checkpoint. Discord/X https://x.com/eliebakouch/status/2078180531456573874 via fxtwitter (quotes Inkling launch). Viz https://eliebak.com/viz/jspace-open. Code/data https://github.com/eliebak/open-jlens-data (9 stars; LICENSE null at check; tensors not in git). Companion HF PrimeIntellect/Laguna-XS.2-jlens. Anthropic method https://transformer-circuits.pub/2026/workspace/ and github.com/anthropics/jacobian-lens. Fitted instrument, not a training corpus, so no datasets_local row. Distinct from papers_local 656 (Inkling model card).