This site uses Just the Docs, a documentation theme for Jekyll.

2.1.12 Caching Model

Motivation

Models may want to cache information when acting as a proxy for data sources that are slow, unreliable, or have an access quota or cost. Typical examples are network services, databases, disk access, or when the information is obtained from long-running computations.

Using a cache generally delivers the following advantages:

  • Gives a faster response after the first access, improving the perceived performance of the application
  • Neutralizes potential temporary failures of the data source, improving perceived reliability
  • When accessing a remote resource, prevents repeated and frequent requests of the resource. Failure to implement caching in this case generally leads to a security response from the administrators of the remote resource (e.g. throttling, or denied access).

However, caching comes with a set of liabilities:

  • Increases memory footprint of the application to store the cached data
  • The Model may return outdated information to its client
  • In multi-user web-applications, cached data may contain critical security information, and must therefore not be leaked to unintended recipients.

Design

The typical design of a Caching Model involves the following steps to retrieve the data from a source:

  1. The View attempts to retrieve some information from the Model.
  2. The Model first attempts to obtain this information from the Cache.
  3. If the desired information is not found, the Model will perform a query to the data source, obtain the result, add it to the Cache, and return it to the View.
  4. Additional attempts to retrieve the same information from the Model will extract data from the Cache.

When parameters are passed during the get request, the cache must have an index over these parameters to guarantee a retrieval of the intended information. For example, if a Model class UserRepository has a method get_user_data(user_id), this method must use the parameter user_id to perform the intended retrieval from the cache.

To perform a set operation on the Model, two strategies are possible. The first one is to apply the changing action on the slow data source, followed by either a similar update or an invalidation of the cached data. This strategy has the following caveats:

  • If the set action is expected to change the data source in a non-trivial way, it may not be possible to perform a sensible change to the cached data with the same business logic supported by the remote source. Instead, one should remove the cached data to promote a fresh retrieval at the next get operation.
  • The operation requires a round trip to the slow data source, with consequences both at the UI level (failures must be reported, in-progress operations need to present a sensible progress bar, perceived performance may suffer) and also at the application design level (if repeated set operations are needed, performing a round trip to the data source every time is wasteful and impacts performance).

The second possible strategy is to update the cached data and postpone the data source update, possibly bundling multiple changes into a single request. With this solution, the cache must not evict the changed information until it has been committed to the data source. Further issues may exist for resources that are shared among clients, or in case of application crash or network failure where the changed content is only partially submitted or not at all.

Clients of the Model may need to be aware of the existence of a cache if they want to guarantee up-to-date data, or need to force flushing of changes to the remote service.

Caching strategies

Choosing an appropriate caching strategy is a complex topic well outside the scope of this book. Several factors may need to be kept into account, such as data access patterns, specific requirements of the design and the application behavior, and data consistency.

To present a few among many possibilities, the most trivial strategy is to simply perform memoization of the result. A Model using memoization effectively uses a cache whose entries never expire. The cache size can be kept under control by discarding the Least Recently Used (LRU) data.

LRU constraint the size of the cache and keeps the most recently used data, but does not provide a predictable expiration strategy, something that may be required for some applications. For example, a News Ticker application retrieving information from a website may want to invalidate the cached data and perform a new retrieval, hopefully to obtain updated information, after a fixed absolute timeout (e.g. 15 minutes). Other applications may prefer a sliding expiration strategy: cached data expire only if not accessed within a given amount of time (e.g. 5 minutes, but the countdown resets to zero if requested again), or when an explicit method call is issued on the Model object. Smarter strategies may use a small payload request to ask the remote service if new or changed data are available, and act accordingly by either retrieving the new data or return the cached information.

Additional optimizations that are frequently used are preemptive caching and priority caching. Preemptive caching stores Model data that is likely to be accessed soon, even when not directly requested. For example, while retrieving information about a given user from a social networking website, a smart caching engine may decide to retrieve its most recent pictures and populate the cache of the picture Model objects; Priority caching allows to consider different levels of likelihood for eviction, removing low priority before high priority data.