GRASP: New Gradient-Based Planner for Long-Horizon Planning in World Models
Berkeley Artificial Intelligence Research (BAIR)
Researchers from BAIR propose GRASP, a gradient-based planner that addresses challenges in long-horizon planning with learned world models. It lifts trajectory optimization to parallel virtual states, introduces stochasticity for exploration, and reshapes gradients to avoid brittle state-input gradients.
GRASP is a new gradient-based planner for learned dynamics (world models) that makes long-horizon planning practical by (1) lifting the trajectory into virtual states so optimization is parallel across time, (2) adding stochasticity directly to the state iterates for exploration, and (3) reshaping gradients so actions get clean signals while avoiding brittle 'state-input' gradients through high-dimensional vision models. Large learned world models are becoming increasingly capable but long-horizon planning remains fragile due to ill-conditioned computation graphs, non-greedy landscapes with bad local minima, and high-dimensional latent spaces introducing subtle failure modes. The authors from BAIR, with Mike Rabbat, Aditi Krishnapriyan, Yann LeCun, and Amir Bar, propose GRASP to address these issues. The planner builds a first-order lifted state approach dependent only on action Jacobians through the world model, exploiting differentiability while avoiding sensitivity of state Jacobians.
- Сокращения
- GRASP = Gradient RelAxed Stochastic Planner — Градиентный релаксированный стохастический планировщик
- BPTT = Backpropagation Through Time — Обратное распространение через время
- LLM = Large Language Model — Большая языковая модель
- RL = Reinforcement Learning — Обучение с подкреплением
Source: BAIR (Berkeley AI) —
original