Why humanoids struggle to grasp moving objects: developing VLA for dynamic environments
Unitree Robotics
Modern VLA models work well in static scenes but fail when objects move, like on a conveyor. The main issues are latency, outdated action chunks, and lack of temporal context. Researchers are now developing whole-body dynamic VLA to handle moving objects and maintain balance.
Most robot demonstrations today occur in static environments with stable lighting and fixed cameras, where VLA models perform well. However, real-world tasks often involve moving objects, such as items on a conveyor belt. When a full-sized humanoid robot interacts with moving objects, it must also walk and maintain balance, complicating motion calculations. The main challenges include latency between observation and action, outdated action chunks, and the inability to determine object speed from a single image. Recent models like DynamicVLA address these issues by updating actions more frequently and considering timing. For humanoids, whole-body control is essential because arm movements affect torso balance. The RLDX-1 architecture processes multiple information streams (motion, memory, physics, action) to improve dynamic manipulation. At MTS Web Services, the author is adapting RLDX-1 to control the Unitree G1 humanoid for whole-body dynamic grasping of conveyor items, treating the task as a closed loop: observe, predict, intercept, execute, verify, correct.
- Сокращения
- VLA = Vision-Language-Action — Vision-Language-Action (зрение-язык-действие)
- IMU = Inertial Measurement Unit — инерциальный измерительный блок
- ML = Machine Learning — машинное обучение
Source: Habr — хаб ИИ —
original