AI Models Can Run on Anything: Enthusiast Runs LLM on a $10 Microcontroller

In 2026, large language models can be run locally on laptops and even smartphones, but an enthusiast under the pseudonym SlvDev went further by running a small language model on an ESP32-S3 microcontroller priced at $10, achieving a processing speed of nearly 10 tokens per second. The ESP32-S3 has 520 KB of SRAM and 8 MB of pseudo-SRAM (PSRAM), and is designed for controlling remote sensors, IoT devices, and other equipment, but running something like DeepSeek V4 Flash is out of the question. Instead, SlvDev chose TinyStories, a model developed by Microsoft Research with 28.9 million parameters, which is about 10,000 times more compact. Even this model initially required about 60 MB of space at 16-bit precision, which was too large for the ESP32-S3. The author therefore quantized the model, compressing its weights from 16 to 8 bits, and then to 4 bits, reducing the model size to 14.9 MB, which required a version of the microcontroller with 16 MB of flash memory. Further, the enthusiast implemented a per-layer embedding (PLE) mechanism used in the architecture of Google Gemma open models, allowing 25 million parameters (12 MB) to be moved to flash memory, which is accessed relatively infrequently, while only 2 MB of data was placed in the controller's RAM. As a result, input headers and key-value (KV) cache were placed in PSRAM, and 520 KB of SRAM sufficed for activation. These measures achieved generation at 9.88 tokens per second, faster than an average person can read. TinyStories is a great concept demonstration but only generates short stories, insufficient even for a chatbot. The similar-sized Barista model can answer questions, generates tokens twice as fast, but only on espresso-related topics. Both models are too small for other tasks, but the fact they run on an ESP32 is impressive.