nanochat

Summary

RiddleHe/nanochat forks karpathy/nanochat into two hackable frameworks: nanochat/ for pretraining architecture variants (boolean flags in gpt_base.py, model_registry.py, torchrun scripts/base_train.py, DCLM CORE eval via scripts/base_eval.py) and nanorl/ for RL objective-function ablations (ALGORITHMS in nanorl/loss.py, vLLM rollout worker synced to a torchrun trainer). Ablations.sh runs d12/d24 FLOP-budget batches and skips existing checkpoints. MIT. Cite: Muyu He, Yuchen Liu 2026 (nanochat-arch).

Keywords

nanochat · karpathy · architecture-ablation · rl · nanorl · dclm-core · small-lm

Topics

small LM pretraining, RL ablations

Research notes

• Primary: GitHub README + API (MIT, Python, 58 stars / 5 forks at check; fork of karpathy/nanochat). Discord posted papers/bank_of_values.pdf; that path and papers/ 404 at check. Cataloged the repo URL as the leftover GitHub item. Training/eval uses public DCLM CORE fixtures in-repo, not a new hosted corpus, so no datasets_local row.