ABBEL: Training LLMs to Update Beliefs for Efficient Long-term Interaction

Researchers from BAIR have introduced the ABBEL framework (Autoencoder Belief Bottleneck for Efficient Learning), which addresses the problem of scaling LLM context during long interactions. Traditional recursive summarization (context compaction) reduces performance, especially in tasks with a scarcity of high-quality data, such as collaborative code generation. ABBEL isolates the summarization generation task by framing it as an update of belief states—textual representations of interaction history. Training uses belief grading: the model is rewarded based on how well the latest observation can be reconstructed from the current belief state. In the CollabBench environment (collaborative programming), ABBEL with reconstructive grading closes the gap with full-context models by approximately 50% and trains twice as fast as without belief grading. In a simulation of the Combination Lock game, the method using domain knowledge-based grading achieves or surpasses full-context model performance. In a multi-task question-answering environment, introducing a Peak Belief Penalty significantly reduces memory usage without substantial quality degradation. ABBEL incurs O(N/K) summarization overhead, where N is the total number of actions and K is the number of actions between summarization runs. The researchers note that future combinations of different memory types (working, short-term, and long-term) could lead to more powerful systems.