Boltzbit and Cambridge propose dynamic weight injection for LLMs via hypernetwork
Researchers from Boltzbit and Cambridge introduced a method to dynamically compile user-provided context into low-rank LoRA weights using a hypernetwork, allowing LLMs to 'write knowledge into weights' instead of relying on context window. This approach shows improved accuracy on long documents and multi-turn conversations with constant compute cost per turn.