How Models Pack Concepts into Manifolds: Theory, Practice, and Searching for Circles

The article explains that AI models pack real-world concepts into geometric structures within their activation spaces. Concepts like binary oppositions form two isolated points (0‑dimensional), days of the week form a circle (1‑dimensional manifold), and hierarchies form simplexes where dimensionality is not applicable. Whitney's embedding theorem (1944) states that any smooth d‑dimensional manifold can be smoothly embedded in R^{2d}, which gives a theoretical upper bound on the required dimensions. However, the theorem does not directly limit models due to approximate embedding and superposition. Manifolds emerge when a concept is continuously ordered and the model's training objective requires operations on that structure. Practical examples include circular representations for days of the week found in GPT-2 (layer 7) and Mistral 7B via SAE clustering and PCA; interventions on these structures alter model behavior.