Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data

Summary

Infinite-Parameter LLMs' (Hu, Clarke, Zhang, Hernández-Lobato, Cambridge) proposes an architecture where effective model weights are generated from live session data by a compact hypernetwork producing low-rank modulations of a frozen base, with a Bayesian belief updated online. Stored parameters stay fixed while compilable weights are effectively unbounded — a structural alternative to keeping all runtime knowledge in the prompt.

Keywords

hypernetwork · continual-learning · architecture · in-context-learning

Topics

hypernetwork, continual-learning, architecture, in-context-learning

Research notes

• Method: Taking inspiration from MoE, a compact hypernetwork turns runtime interaction data into a low-rank modulation of a shared base network's feed-forward weights; a Bayesian belief over the generator's latent code is updated online per session.
• Key findings: Proposes a compact hypernetwork that converts live session data into low-rank modulations of a frozen base model — effective weights generated from data, not stored; Bayesian belief over the generator's latent state updated online as the session proceeds, so weights are re-derived from an evolving belief rather than frozen after one read; Argues weight-carried knowledge is amortized in compute, frees context, persists across turns, and can generalize better than in-context use; Specifies an evaluation protocol testing the approach against in-context learning and retrieval
• Limitations: A research proposal: no production deployments, no third-party replications, and results not yet independently verified (per third-party coverage). v2 revised Sep 21, 2026.
• University of Cambridge. 50KB v1 (Sep 16, 2026); 55KB v2 (Sep 21). Subjects: cs.AI, cs.LG.