Google engineers Todd Lipcon and Manish Purohit have introduced linear elastic caching, a technique for minimizing cloud infrastructure costs by dynamically adjusting cache memory size based on workload demands. The approach treats memory as a utility where costs accrue based on size and duration, drawing on the "ski rental" problem to determine whether to retain data in RAM or evict it to avoid ongoing memory fees.

Traditional cache management relies on fixed memory allocations and static eviction policies like least recently used (LRU) replacement, which often results in inefficient resource usage. The linear elastic caching method uses a machine learning model to assign an optimal time-to-live (TTL) value to cached pages for each request. This allows systems to balance memory footprint against the potential latency penalty of a cache miss.

Testing within Google’s globally distributed database, Spanner, demonstrated significant operational gains. The implementation reduced memory usage by 15.5% while increasing cache misses by only 5.5%. Consequently, the total cost of ownership (TCO) for the cache infrastructure dropped by approximately 5%. Because the algorithm is cost-aware, the slight rise in cache misses primarily involved data that is inexpensive to retrieve, resulting in a negligible 0.5% impact on overall I/O costs.

To ensure applicability beyond Google’s internal infrastructure, the team evaluated the approach against public cache traces using an optimized implementation of the greedy dual size frequency (GDSF) policy as a baseline. By using lightweight, shallow decision trees for prediction, the system remains performant even when handling billions of requests per second. The research suggests that applying simple machine learning models to core infrastructure can significantly improve economic efficiency as cloud environments shift toward granular, pay-as-you-go pricing models.

Google engineers Todd Lipcon and Manish Purohit have introduced linear elastic caching, a technique for minimizing cloud infrastructure costs by dynamically adjusting cache memory size based on workload demands. The approach treats memory as a utility where costs accrue based on size and duration, drawing on the "ski rental" problem to determine whether to retain data in RAM or evict it to avoid ongoing memory fees.

Traditional cache management relies on fixed memory allocations and static eviction policies like least recently used (LRU) replacement, which often results in inefficient resource usage. The linear elastic caching method uses a machine learning model to assign an optimal time-to-live (TTL) value to cached pages for each request. This allows systems to balance memory footprint against the potential latency penalty of a cache miss.

Testing within Google’s globally distributed database, Spanner, demonstrated significant operational gains. The implementation reduced memory usage by 15.5% while increasing cache misses by only 5.5%. Consequently, the total cost of ownership (TCO) for the cache infrastructure dropped by approximately 5%. Because the algorithm is cost-aware, the slight rise in cache misses primarily involved data that is inexpensive to retrieve, resulting in a negligible 0.5% impact on overall I/O costs.

To ensure applicability beyond Google’s internal infrastructure, the team evaluated the approach against public cache traces using an optimized implementation of the greedy dual size frequency (GDSF) policy as a baseline. By using lightweight, shallow decision trees for prediction, the system remains performant even when handling billions of requests per second. The research suggests that applying simple machine learning models to core infrastructure can significantly improve economic efficiency as cloud environments shift toward granular, pay-as-you-go pricing models.