Recirculation

Summary

An inference-time architectural enhancement for off-the-shelf foundation models that markedly reduces perplexity and boosts accuracy across generation and reasoning tasks. Recirculation introduces a specific form of recurrence that lets the model act as a dynamical system tracking belief states, motivated by the limitation that state updates in feedforward transformers are bounded by model depth. It adds essentially no generation latency (serial processing only in the prefill phase), and an adaptive variant needs only light hyperparameter tuning with frozen model weights.

Keywords

inference · recurrence · architecture · test-time · belief states

Topics

inference, recurrence, architecture, test-time, belief states

Research notes

• Method: Training-free inference-time recurrence ('recirculation') turning the model into a dynamical system for belief-state tracking; adaptive variant tunes hyperparameters while freezing original weights. Distinguished from chain-of-thought (reserved for complex inference) and from depth-recurrence/looping techniques.
• Key findings: Systematic perplexity reduction on a suite of datasets for the Gemma3 family; 21% accuracy increase on GSM8k vs the off-the-shelf baseline; Reliable accuracy improvements on other downstream tasks with no additional training
• Limitations: Requires serial processing in the prefill phase; demonstrated on the Gemma3 family — generality to other model families not shown in the abstract. Paper under CC BY-NC-SA 4.0.
• Original Discord link was the /pdf/ form.