Boltzbit and Cambridge: dynamic weight updates for LLMs
Boltzbit and Cambridge University researchers developed a method to write knowledge into model weights instead of prompts. A hypernetwork dynamically generates LoRA weights from user interactions, enabling continuous learning without retraining. Experiments show improved accuracy on long documents and multi-turn conversations with constant compute cost.