XNews

Jack Rae (OpenAI) Frames Large Language Model Training as Lossless Compression, Citing Llama Numbers

The post presents Jack Rae's perspective that training large language models is equivalent to lossless compression: better compression = deeper understanding. It cites Solomonoff's theorem, the Chinese Room thought experiment, and Llama 65B compressing 5.6TB to ~400GB (14×), argues model weights need not be shipped, and frames compression as an un-gamable training objective with implications for scaling, architectures, tools, multimodal fusion and AGI limits.

Source posts