Jack Rae / OpenAI concept: Training large language models is lossless compression of data
The post explains Jack Rae’s OpenAI framing that training large language models is essentially lossless compression: better compression = deeper understanding. It links this to Solomonoff theory, explains why larger models generalize, shows numeric compression examples (Llama65B → 5.6TB to ~400GB), and argues compression defends against cheating and guides AGI research.