Needle 2: An Open 45M-Parameter Tool-Calling Model in a 14MB Binary

Cactus Compute has released Needle 2, an open 45M-parameter model for tool calling, device use, and structured extraction. The entire model ships as a single 14MB binary that runs a full session in about 28MB of RAM. Weights are trained and deployed at CQ2-bit using Cactus Quants, and the model is sealed inside the company's own C++ engine, so there is no runtime to install and no download at inference time. Reported decode throughput is 500 tokens/sec on a Raspberry Pi 5, 400–1,500 tokens/sec on Meta Quest 3S and Apple Vision Pro, and 300–700 tokens/sec on sub-$200 phones. The design premise is that mapping a messy sentence onto a typed function signature needs no world knowledge and no open-ended prose, so 45M parameters are enough. Needle 2 uses a Simple Attention Network architecture with a Hadamard MLP, GQA attention, engram key-value memory, and multi-lane hyper-connections. It uses a 256-token sliding window and pinned KV sinks to keep memory near 28MB regardless of conversation length. The model includes a contrastive retrieval head that selects only the top five tools when more than five are declared, and every response carries a confidence value. Evaluation on public function-calling benchmarks shows Needle 2 leads both Seal-Tools splits and posts 98.3 function-name accuracy on Mobile Actions, but trails on BFCL v4 overall, which Cactus attributes to distribution differences.