Software Caching for Fault Tolerance

Software that caches data often gains from decreased latency. However, this not the only use for caching: caching loosens the coupling between different systems on a network, making failures in the data source less likely to also drag down the component that caches.

Operating systems often include caching software: nscd(8) and equivalent. These may not provide much resiliency during outages. A user debugging a problem may not exist in the cache, and therefore cannot login when the directory service is unavailable or unacceptably slow. DNS likewise may or may not have a cached entry, though this is more likely for hosts frequently looked up than a user who would only login when problems occur. In the past, I have run stealth authoritative caching nameservers on critical systems: this provides both a cache should the primary name servers be unavailable, and also an authoritative answer for certain records, eliminating the need to check elsewhere.

In-house software can benefit the most from proactive caching, as this software can be tuned to business needs. Consider host metadata stored in a LDAP service. During normal operations, a client can probably make redundant queries with no ill effect, or daily backups of the LDAP service will cause no problems. However, should a data center fail, and everyone begin querying the LDAP service for host status (while the backups are running), the incident will become far worse if LDAP crumples under the load. If instead client systems appropriately cache the host metadata, that metadata will still be available during a major outage—with the caveat that potentially stale data might be in use.

The Perl example below illustrates simplistic memory caching via memoization. Subsequent lookups on a host already stored in %cache will result in a cache hit. This makes the script more efficient and less coupled to the LDAP service, and also removes load from the LDAP service.

{ my %cache = (); sub get_host_metadata { my $self = shift; my $host = shift; if ( !exists $cache{$host} ) { remark( 'DEBUG', 'cache miss', { host => $host } ); $cache{$host} = $self->query_ldap($host); } else { remark( 'DEBUG', 'cache hit', { host => $host } ); } return $cache{$host}; } }

This simplistic caching can be improved on. Options for Perl include:

• tie the %cache via DB_File. This persists the data between different instances of the script, and avoids the edge case of %cache consuming all available memory.
• Use Cache::FileCache, or any of the related Cache::* modules, which offer many convenient features, such as item lifetime and expiration.
• Persist the data with memcached. Multiple systems can then share and update the same cache.

While many applications will benefit from caching, some may not. The extra work of checking a cache, versus directly obtaining the result, may be slower. Analyze the cache hit versus miss ratio, and the duration of the cache hit, miss, and direct access calls. Also consider failure cases: what is the likelihood that the data source will fail or degrade to an unacceptable level of performance? Even if caching is slower than direct access, how critical is the data? A cache will make the data more available.