NVIDIA AI Releases Molt: A PyTorch-Native Agentic Reinforcement Learning Framework

NVIDIA's NeMo team has released Molt, a PyTorch-native agentic reinforcement learning framework designed to reduce the research overhead of constantly modifying RL algorithms. The codebase is intentionally compact—roughly 8.6K lines of RL code, compared to about 62K for verl and 25K for slime—so researchers and AI coding assistants can grasp it entirely. Molt is deployable under Apache 2.0 with launch codes, Slurm scripts, and a prebuilt container, but the paper positions it as research infrastructure, not a production service. The shipped recipes assume 2 nodes of 8 H100 GPUs, split with 8 for training and 8 for rollout, putting it in reach of frontier labs, well-funded startups, enterprise research groups, and academic labs with multi-node H100/H200 access. Molt composes Ray for placement and queues, vLLM for rollout, and NVIDIA AutoModel with FSDP2 for training—none forked, so upstream improvements arrive as a container pin. The runtime uses an agent pool of vLLM engines behind a request router and a single trainable policy actor, with partial rollout pausing engines and broadcasting actor shards via NCCL. An RL run exports an AgentRunner module, supporting Env (Gymnasium-aligned step()) or ChatAgent (user-owned loop via OpenAI or Anthropic SDK) through a loopback server that ensures token-exact accumulation. Design correctness invariants include token identity, policy-version semantics, and forward consistency; for mixture-of-experts policies, Molt applies rollout routing replay where vLLM returns per-token expert ids and the training forward replays them. Throughput is statistically comparable to a Megatron-based stack, with the MoE-mismatch caveat disclosed, and the same loop runs a dense 4B model or a 700B MoE at --fsdp.ep_size 256.