No Magic: 35 Controlled Experiments Show That "Exotic Algebras" Do Not Outperform Properly Tuned Real-Valued Architectures

Researchers studied the effectiveness of hypercomplex algebras (quaternions, octonions, sedenions, Clifford algebras) in neural networks through 35 controlled experiments. According to them, typical comparisons in the literature are flawed: hypercomplex layers are compared with unstructured baselines, while the advantage might simply come from having any structure. The authors introduced four control architectures with the same number of parameters: Real-rotation, Shuffled structure tensor, Real HRR, and Real-orthogonality. After adding at least one control, the claimed advantage disappeared in all 8 original tasks, including their own headline result with an octonion state-space model — it only outperformed on groups embeddable in quaternions. The sole exception was a state tracking task on the binary tetrahedral group, where quaternion recurrence extrapolated to length 1024 with accuracy 0.992. However, this is explained by the optimization landscape rather than algebraic magic: SO(4) contains an exact solution, but stochastic gradient descent reliably finds it only within the quaternion parametrization. Moreover, LRU (complex diagonal recurrence) beat the quaternion version with the same number of parameters. The authors also proved a series of exact impossibility results: reflections are unattainable in Cayley–Dickson algebras, zero divisors cannot implement 3-way exclusion (max 0.500, vs. 16-way for orthogonality), non-associativity cancels parallel scan for long sequences, and tensor products of algebras collapse into ordinary matrix algebras. Ultimately, the best result came from the 'boring' two-dimensional complex algebra. The authors advocate mandatory use of param-matched real structured control and shuffled-structure control in research on structured layers to avoid false positives. The preprint and code are publicly available.