Mistral AI Unveils Robostral Navigate: A Model for Robot Navigation with a Single RGB Camera
Mistral
Mistral AI introduces Robostral Navigate, an 8B parameter model that enables robots to navigate complex environments using only a single RGB camera. It achieves 76.6% success on unseen R2R-CE benchmarks, outperforming multi-sensor approaches while being more efficient.
Mistral AI has unveiled Robostral Navigate, its first model built for embodied navigation. The 8B parameter model takes RGB images and plain-language instructions to move a robot through environments. Unlike other models that use depth sensors, LiDAR, or multiple cameras, Robostral Navigate operates with a single ordinary RGB camera and no depth sensors, achieving 76.6% on R2R-CE (Room-to-Room in Continuous Environments) validation unseen. This beats the best single-camera approach by 9.7 points and the best system using depth or multiple cameras by 4.5 points. The model is built entirely in-house with simulated data and token-efficient techniques, and generalizes across wheeled, legged, and flying robots. It uses a pointing-based navigation method that predicts image coordinates of target locations, falling back to local coordinate frame displacements when targets are out of view. Training efficiency is achieved via prefix-caching, reducing training tokens by 22×; online reinforcement learning with CISPO further improves success rate by 3.2%.
- Сокращения
- R2R-CE = Room-to-Room in Continuous Environments
- RGB = Red Green Blue
- LiDAR = Light Detection and Ranging
- VLM = Vision-Language Model
- CISPO —
Source: Mistral AI —
original