Boltzbit and Cambridge introduce Hypernetwork-based infinite parameter LLM
Boltzbit and Cambridge University researchers proposed a new method for LLMs that dynamically compiles user interaction data into low-rank weights via a lightweight hypernetwork, instead of freezing weights after training. A Bayesian mechanism allows these weights to evolve during conversation, enabling continuous parameter generation. Experiments showed this approach outperforms traditional prompt-based methods on long documents, multi-distractor, and multi-turn dialogue tasks.