Inside Apple Neural Engine in M4, Part 1: Reverse Engineering

The article details how the authors reverse-engineered Apple's Neural Engine (ANE) in the M4 chip, bypassing Core ML to work with the hardware directly. They explored the full software stack from Core ML to the IOKit kernel driver, discovered how to compile and execute programs on ANE without Core ML, and unmasked the binary format. They measured the actual peak performance, finding that Apple's advertised '38 TOPS' is misleading. Ultimately, they managed to train a neural network on the chip, which is intended solely for inference. The ANE is not a GPU or CPU; it is a computational graph engine that executes a compiled neural network as a single atomic operation. They used method swizzling and binary analysis to uncover over 40 private classes in AppleNeuralEngine.framework, including _ANEClient, which allows direct access to the compile-load-execute pipeline. They also discovered that the MIL language is used to describe models, and the binary format E5 is a FlatBuffers file that parameterizes fixed hardware primitives rather than storing traditional machine code. They found a way to compile models in memory using _ANEInMemoryModelDescriptor, though it still writes to a temporary directory. The ANE in M4 has 16 cores, a queue depth of 127, and independent dynamic voltage and frequency scaling (DVFS).