Archive for August, 2011

Filtering effects in cache hierarchies

Posted in Uncategorized on August 31, 2011 by swaminathans

A friend of mine was building a multi-tiered application with a cache fronting a database and was investigating why his database memory hit ratio was not as high as he expected it to be and how to optimize it. This matters only when you are paranoid about latency percentiles at TP99 or TP99.9s. One of the things, he found after a deeper investigation is what he called as “cherry picking” done by the caching tier that the next level did not see requests with high cache locality. A similar observation was made in Web cache research and was referred to filtering effects.

One of the important works done in this area is by Carey Williamson on “On Filter Effects in Web cache hierarchies”. This paper uses trace driven simulation to look into effects of what happens to workload locality when it travels through various cache hierarchies. Important results to call out:
– Proxy caches filter out the most popular documents from a workload. The filtering effect typically changes the Zipf-like document popularity profile of the input stream into a two-part piecewise-linear document popularity profile for the output stream. The upper leftmost part of the plot is significantly flattened.
– The percentage of unique documents referenced and the percentage of onetimer documents tend to increase as the request stream passes through a cache.
– Among the caches in a caching hierarchy, the first-level cache has the most pronounced filtering effect on the workload.

Even though this is a 10 year old work, highly recommended read for folks trying to understand the effects on caching on their systems.

Amazon ElastiCache

Posted in Uncategorized on August 23, 2011 by swaminathans

Last night, my sister team in AWS launced a service I’m very excited about: Amazon Elasticache. Historically, caching has been the one of the most widely used techniques to build scalable web applications where the caches store the most often accessed computation results which take longer (or is harder) to re-compute in the source. In-memory caches are normally used to front databases so that often accessed results can be retrieved from memory faster (see examples of how to use MySQL and memcache together, here). However, to ensure that in-memory cache do not become a scalability bottleneck themselves, distributed cache clusters use techniques like distributed hash tables (DHTs) to ensure that cache cluster can be “scaled out”. As the scale of caching system becomes harder, it is a challenge to manage them in a large scale environment.

Today, AWS has made the process of running a cache cluster easier with a new managed cache offering called Elasticache. A quote from the detail page sums it up well:

“Amazon ElastiCache is a web service that makes it easy to deploy, operate, and scale an in-memory cache in the cloud. Amazon ElastiCache is protocol-compliant with Memcached, a widely adopted memory object caching system, so code, applications, and popular tools that you use today with existing Memcached environments will work seamlessly with the service. Amazon ElastiCache simplifies and offloads the management, monitoring, and operation of in-memory cache environments, enabling you to focus on the differentiating parts of your applications.”

Congratulations team!