NVIDIA Releases Molt, Open-Source PyTorch-Native RL Framework for Agentic Research

NVIDIA released Molt, an open-source PyTorch-native reinforcement learning framework for agentic research built on Ray, vLLM, and NVIDIA AutoModel with FSDP2. The framework defines rewards through Python functions in Env or ChatAgent interfaces and scales from 8B models to 1T-class MoE with the same training script.
Event version 1.
Key insights
Minimal Three-Component Stack
Molt's entire stack consists of Ray for placement and async queues, vLLM for rollout, and NVIDIA AutoModel with FSDP2 for training. The stack is intentionally minimal with one trainable actor and optional KL reference workers.
Python-Defined Rewards Without Pretrained Reward Models
Rewards in Molt are any Python function, including mathematical graders, LLM-as-judge, multi-turn tool evaluators, and VLM environments. Users define what good means inside an Env or ChatAgent class, and no pretrained reward model is required.
Two Agent Interfaces
The Env interface is Gymnasium-style, with the framework owning the LLM loop and step() returning a reward. The ChatAgent interface gives the loop via the OpenAI or Anthropic SDK, so external harnesses speaking those APIs work without modification.
Scaling From 8B to 1T-Class MoE
The same script that trains an 8B model scales to 1T-class MoE. DeepSeek-V3-class actors run at ep_size 256 with TP, EP, and CP parallelism plus Adam CPU offload, with no rewrite between scales.
Supported Algorithms
Molt supports REINFORCE, REINFORCE baseline, RLOO, GRPO, DR-GRPO, GAE (PPO), and on-policy distillation. Rollout, training, and weight sync overlap fully asynchronously.