Caching

A cache allows applications to store results from expensive data operations (e.g., a database call or web service request) so they can quickly accessed if requested multiple times.

Hierarchy

The Verint cache object is built by combining one or more layers together to form a single logical structure. Each layer sits atop another, creating a hierarchy that defines how requests are satisfied:

When a retrieval request comes in, each layer is searched until the item is found. If the item is found, that item is copied to all layers above the cache that satisfied the request. This allows cached objects to move from slower caches to faster caches, maximizing locality principles and reducing global system stress.

Providers and scope

Each layer in the hierarchy defines how it caches objects. A layer specifies its supported caching behavior by using one or more CacheScope values. Only requests containing the supported scope will be sent to a provider. 

CacheScope values are:

The Verint cache object comes with built-in providers that are available for use. They include:

• HttpContext (Context scope): Items are cached inside the current HTTP request. This is the fastest cache available for web requests and should always be designed at the lowest level cache in a website environment.

• ASP.NET (Process scope): Items are cached inside the HttpRuntime object. Items cached using this provider are available to all threads that access the same HttpRuntime object. 
  
       NOTE: This cache can be used outside of a web request.

• AppFabric (Distributed scope): Items are cached using the Microsoft AppFabric distributed cache system. Items in this cache are available to all processes that can access the AppFabric server.

Default timeout

When items are placed into cache, it is not necessary to give a timeout. In these cases, the currently configured default timeout value is used. The out-of-the-box default timeout is 1 minute and 40 seconds.

Many objects in Verint Community Server use the default timeout value. In addition to a very simple scaling mechanism, controlling the default timeout allows for finer control over the cache system in webfarm scenarios where staleness is a concern. Generally, if more than one server is running a site, the default timeout value should be low to keep staleness at a minimum. The lower the value, the less likely you are to see staleness - but at the cost of decreased cache performance.

Default timeouts are set inside of the caching configuration XML settings. For more information, please see the Configuration documentation.

Cache factors

Cache factors allow us to complete the picture of how long items should be cached. Cache factor values can be set for any cache layer and is used as a universal adjustment on timeout.

Where cache factors become important is when a cache is set up using a distributed cache. For example, assume we have a website that contains many nodes (webfarm). Each node is running the AspNetCache provider to cache items locally on each server. Because we're in a webfarm scenario, we have to reduce our default timeout because we can't risk staleness and other cache inconsistencies.

To increase the cache performance of the website, we want to introduce a distributed cache so longer-lived objects can be held in memory for longer amounts of time. In this situation, we want to increase the default timeout because we know that all web nodes will keep the distributed cache in sync as long as we relay the changes back to the distributed cache server. But this poses a problem: If we increase the default timeout, then the locally cached items in the AspNetCache will risk being stale in any web server that doesn't handle the data operation. But if we do not increase the default timeout, objects will be removed from distributed cache much more frequently than they need to be.

To solve this problem, we can use a cache factor on the distributed layer. By setting a cache factor on the distributed layer, we can keep a lower default timeout for our local server caches. Items sent to distributed cache will be cached longer, by default, then items cached on the local server.

It should be noted that all timeouts are subjected to the factor, not just default timeouts.

Cache factors are set inside of the caching configuration XML settings. For more information, see the Configuration documentation.

Overrides

Included in the Verint cache system is a mechanism allowing runtime adjustments to be made to cache operations. This allows users to modify values used in Community Server (or added by a third-party component) if they have a site-specific need to do so. Overrides should be used sparingly, but they offer a level of control that may yield a large benefit.

Before an item is sent to any cache layer, conditions are checked to see if a particular override should be applied. If a condition is met, that override is triggered and no further conditions are processed. This means that only one override can be executed for each cache operation.

Conditions can be defined using any of the following constructs:

Additionally, conditions can be combined to form composite conditions. Composite conditions include:

Overrides are defined declaratively using XML. For more information on how to set and configure overrides, see the Configuration documentation.