How AI Agent Memory Prevented a False Report and Forced Abandonment of Its Own Feature
An AI agent building a memory system called vecmory twice avoided harmful actions by recalling its own past notes. The first case stopped a router with an inflated 0.98 correlation that was useless on real data; the second forced the team to drop a beloved feature (node importance) after measuring it lowered retrieval quality. The key lesson: memory that contradicts the author is more valuable than memory that merely echoes.
The vecmory team builds a semantic memory for AI agents, centered on three operations: recall, remember, and link. The agent itself writes development notes into its own memory. In one session, it proposed a router to choose between cosine and graph-based ranking, showing a synthetic correlation of 0.98. Before reporting success, it recalled a previous note revealing that the same idea had failed on real data because synthetic embeddings falsely separate similar from dissimilar — real cosine distributions overlap heavily. The agent canceled the report. In a second case, the default ranking mixed cosine with node in-degree (importance). Measurements on real query-answer pairs showed pure cosine had MRR 0.81 while the smart blend achieved only 0.41, because global popularity drowned rare specific facts. The team removed importance from the default, keeping graph PPR as an option. The agent's memory thus disproved its own cherished feature. The team also measured causal recall on a real GitHub issue repo: pure cosine found the correct PR in 38% of cases, while traversing cause-effect graph edges raised that to 87%. The article concludes that memory is most valuable when it contradicts its author, not when it reinforces assumptions.
- Сокращения
- MRR = Mean Reciprocal Rank — Средний обратный ранг
- PPR = Personalized PageRank — Персонализированный PageRank
- PR = Pull Request — Запрос на слияние
- API = Application Programming Interface — Интерфейс программирования приложений
Source: Habr — хаб ML —
original
