Open Dreamer: Open-Source Implementation of Dreamer 4 Pipeline in JAX/Flax with Full Training Recipe

Researchers Francesco Sacco, Diego Martí, and Edward Hu, sponsored by Reactor, released Open Dreamer, an open implementation of Dreamer 4 in JAX/Flax NNX. Two repositories were released: one for training (tokenizer, dynamics model, rollout, FVD scoring) and one for local inference. A browser demo streams a generated Minecraft world and allows toggling between real game and world model. The architecture uses a block-causal transformer backbone for both tokenizer and dynamics model. The tokenizer is a transformer-based Masked Autoencoder achieving roughly 100× compression without KL or adversarial loss. The dynamics model (1.6B parameters) performs next-frame prediction using diffusion forcing, flow matching, and shortcut models, and also predicts the next action. Training runs for 200,000 steps with Muon optimizer, WSD schedule, peak learning rate 3e-4, and EMA decay 0.999. The team achieved 57-58% model FLOPs utilization on B200 GPUs, with model state fitting in ~24 GiB. Activations were the main memory cost, solved by data parallelism and activation checkpointing. Stability issues were the biggest challenge; six fixes include Muon optimizer, mandatory EMA weights, careful mixed precision, v-space loss weighting, minibatch barycentric optimal transport, and deemphasizing μ-parametrization. The repository does not include the behavior-cloning or RL training loop, and FVD scores are not published.