Why Even a Perfect LLM Won't Make Agentic Systems Predictable
Beijing Academy of Artificial Intelligence (BAAI)
Even if an LLM stops hallucinating and becomes deterministic, multi-agent systems based on it can remain unpredictable due to nonlinear agent interactions. The author demonstrates with examples that semantically similar queries can lead to opposite responses, and in agent chains, small deviations accumulate and generate deterministic chaos.
The article examines the unpredictability of LLM-based multi-agent systems even under zero hallucinations. The author conducts an experiment with the Qwen Instruct 30B model at temperature 0: for eight similar questions about LLM determinism, the model gave two opposite answers, although each individual query reproduced identically. The semantic similarity of the questions, measured via the BAAI/bge-m3 metric, was 0.30–0.39, meaning they are practically indistinguishable to the user. Next, the author simulates the behavior of an agent chain using a logistic map—a nonlinear transformation of states. With initial values corresponding to semantic similarity, trajectories diverge chaotically by step 15, even though each step is deterministic. Conclusion: predictability of a single agent does not guarantee predictability of the composition due to nonlinearity and error accumulation. Practical recommendations include limiting the length of sequential chains, introducing verification points, using parallel structures, and testing the pipeline on semantically similar inputs.
Source: Habr — хаб ИИ —
original