Linear Elastic Caching: A New Approach to Cache Management Optimizes Cloud Costs

Researchers from Google Cloud and Google Research have introduced linear elastic caching, a new cache management method that minimizes total cost of ownership. Modern high-performance databases and cloud services use in-memory caching for fast data access, but memory costs are high (up to $3 per day for 1 GiB). Traditional fixed-size caching faces the issue: too small a cache degrades performance, too large leads to unnecessary expenses. Elastic caching dynamically adjusts cache size, treating memory as a resource with linear cost depending on volume and data storage duration. The problem is solved using the ski rental algorithm, where for each data fragment, a choice is made between "renting" (storage with time-based payment) and "buying" (long-term storage). A shallow decision tree is used to predict the optimal time-to-live (TTL) for a page, compiled into a few lines of C++, considering data size, cache miss cost, and operation type. Deployment in production Spanner servers over several months showed cache cost reductions of up to 35% with only a 2.6% increase in cache misses, and due to accounting for miss costs, the actual impact on input/output was just 0.5%. Tests on public traces using various algorithm variants (including training on the first half of the trace) also confirmed the advantage of the elastic approach over fixed caches, especially under high memory cost. This work was done in collaboration with Tamas Sarlos and Ravi Kumar (Google) and presented at the CIDR 2025 conference.