The Hidden Grammar of Language Models: Three Discoveries About Text Generation Structure
OpenAI
The author conducted a two-year study of the internal structure of language models and found that text generation follows stable grammatical paths, models build an internal map of sentence direction, and uncertainty arises only at later stages of specific word selection.
In an article on Habr, the author shares the results of a two-year study of the hidden grammar of large language models. The first discovery: text generation is much more structured than it seems—the model moves along a stable core of highly predictable grammatical paths, like main roads in a city, and this structure persists even at different temperature settings. The second discovery: models build internal maps of sentence direction. Experiments with GPT-2 showed that the signal about the grammatical role of the next word appears in the middle of the network, not at the last layer, indicating that the overall direction of the sentence is planned before specific words are selected. The third conclusion: uncertainty arises late, when choosing the exact word, not at the stage of building the grammatical structure. This suggests two complementary processes—one organizes the sentence structure, the other selects words. The author believes these findings will help create more understandable and computationally efficient AI systems.
Source: Habr — хаб NLP —
original