Perplexity and NVIDIA Develop JOLT Post-Training Method, Beats GRPO on Five Benchmarks

Perplexity and NVIDIA developed JOLT, a post-training method in which one model acts as both teacher and student, with the teacher trained on outcome rewards and penalized for diverging from the student. On MATH, JOLT matched GRPO's accuracy with 6.5x fewer training tokens, and JOLT+ beat GRPO on all five benchmarks across math, coding, tool use, and terminal tasks, including 9.9 points on AppWorld.
Event version 1.
Key insights
Teacher and student share one model
JOLT is a post-training method where one model acts as both teacher and student. The teacher receives extra information such as a verified solution that the student does not have.
Teacher trained on outcome rewards
Self-distillation trains only the student, and its teacher can exploit shortcuts unavailable to the student. JOLT also trains the teacher on outcome rewards from its own attempts, with a penalty when its predictions diverge from the student's.
Token efficiency on MATH
On the MATH benchmark, JOLT matched GRPO's accuracy with 6.5x fewer training tokens.
JOLT+ beats GRPO across five benchmarks
JOLT+ supplements JOLT by providing the student model outcome-based rewards in addition to teacher supervision signals. Across math, coding, tool use, and terminal tasks, JOLT+ beat GRPO on all five benchmarks, including 9.9 points on AppWorld.