INTRODUCTION

diginsight SmartCache provides hybrid, distributed, multilevel caching based on age sensitive data management.

• SmartCache is hybrid as it caches data in-memory and on external RedIs databases.
  In-memory cache ensure 0-latency for most recently used data and ensures low pressure (and reduced cost) on the external RedIs database.

• SmartCache is distributed as cache entries on different nodes of a multiinstance application are sinchronized automatically, to avoid flickering of values when querying the same data on different nodes.

• SmartCache is based on age sensitive data management as cache entries are returned based on a requested MaxAge parameter.
  Data is returned from the cache if the requested MaxAge is compatible with the cache entry.
  Otherwise data is requested to the real data provider.
  This allows requesting data with different MaxAge criteria, according to the specific application condition.
  Data loaded by any request, is made available for the benefit of further requests (as long as compatible with their MaxAge requirement).

• SmartCache is Multilevel: The same entries can be cached in multiple levels (frontend, backend or further levels).
  At any level, data is returned from the cache if the requested MaxAge is compatible with the cache entry. otherwise data is requested to the further levels.
  In case all levels entries contains old data, incompatible with the request MaxAge requirement, data is requested to the real data provider.

• SmartCache is Optimized: as:

  • Privileges In-memory cache => it is faster as in memory cache hits are '0-Latency'
  • Minimizes use of external backing storage (e.g. RedIS) => it is cheaper and scalable as accesses to the backing storage are minimized
  • Replicas synchronize always keys and small values, bigger values are synchronized on demand
  • SmartCache supports data preloading and automatic invalidation of the cache entries so, data load latencies can be cut since the first call.
    
    

SmartCache supports caching data with low cost and high performance.
In particular, 0 latency is ensured on in-memory cache hits. also, pressure on external RedIS resource is low as most frequently used entries are managed in-memory.

Also, 0 latency can be obtained since the first and for every call by means of Cache Preloading and Cache Invalidation.

the following image illustrates the five SmartCache tenets:

ADDITIONAL INFORMATION

Using Smartcache the following events are involved when interacting with data:

• Cache hit or cache miss:
  • a cache hit: occurs when a cache entry exists with key and age compatible with the requested data.
    In case the cache value is taken from the External (RedIs) backing storage, we call it a hybrid cache hit.
  • a cache miss: occurs when no cache entry exists for the key or its age is older than requested MaxAge for data.
• Miss notification: every time a cache miss occurs, all instances are notified about it so that in case they receive a request for the same key, they can obtain the value from the instance that owns it, without need to retrieve it from the server again.
• Entry eviction: every time the in-memory cache eccedes the configured quota older and bigger entries are evicted, and off-loaded to the external (RedIs) backing storage.
• Entry invalidation: specific application conditions, may requires cache entries to be invalidated. Cache keys can be marked implementing interface IInvalidatable are notified every time Invalidate action is triggered so that they can be evicted when needed.
• Entry (re)load: a cache key can be assigned a reload delegate so that when invalidation happens, the value is reloaded, to avoid the cache miss latency on the next incoming call.

The following image illustrates the described SmartCache events:

The following paragraph:
STEPS TO USE SMARTCACHE
discusses basic steps to start using Diginsight.SmartCache.

STEPS TO USE SMARTCACHE

STEP 01: add a reference to Diginsight.SmartCache

In the first step you can just add a Diginsight.SmartCache reference to your code:

In case of multiinstance applications Diginsight.SmartCache.Externalization.ServiceBus may be needed to support instances synchronization. In case of AspNetCore applications Diginsight.SmartCache.Externalization.AspNetCore may be useful to support dynamic MaxAge specification from http request headers.

STEP 02: register SmartCache services into the startup sequence

SmartCache services and default settings must be registered into the startup sequence ConfigureServices methdod.
The code snippets below are available as working samples within the smartcache_samples repository.

public void ConfigureServices(IServiceCollection services)
{
    ...

    services.ConfigureRedisCacheSettings(configuration); // reads RedIs connection string

    ...

    // configures Diginsight:SmartCache config section with default settings
    // supports Dynamic-Configuration for MaxAge, expirations etc
    services.Configure<SmartCacheCoreOptions>(configuration.GetSection("Diginsight:SmartCache"))
            .PostConfigureClassAwareFromHttpRequestHeaders<SmartCacheCoreOptions>();

    // adds smartCache services (ISmartCache, ICacheKeyService and other internal services)
    SmartCacheBuilder smartCacheBuilder = services.AddSmartCache().AddHttpHeaderSupport();

    // reads ServiceBus configuration so support instances synchronization
    IConfigurationSection smartCacheServiceBusConfiguration = 
                          configuration.GetSection("Diginsight:SmartCache:ServiceBus");
    if (!string.IsNullOrEmpty(smartCacheServiceBusConfiguration["ConnectionString"]) &&
        !string.IsNullOrEmpty(smartCacheServiceBusConfiguration["TopicName"]))
    {
        smartCacheBuilder.SetServiceBusCompanion(
                sbo =>
                {
                    smartCacheServiceBusConfiguration.Bind(sbo);
                    // add a GUID as a service bus subscription
                    sbo.SubscriptionName = SmartCacheServiceBusSubscriptionName; 
                }
            );
    }

    services.TryAddSingleton<ICacheKeyProvider, MyCacheKeyProvider>();

}

The image below shows Diginsight.SmartCache settings with default MaxAge and Expiration values for cache entries.

"SmartCache": {
    "MaxAge": "00:05",
    //"MaxAge@...": "00:01",
    //"MaxAge@...": "00:10",
    "AbsoluteExpiration": "1.00:00",
    "SlidingExpiration": "04:00",
    "ServiceBus": {
    "ConnectionString": "", // Key Vault
    "TopicName": "smartcache-commonapi"
    }
}

NB.

• ServiceBus configuration is required only in case of multiinstance applications where instances cache entries need to be synchronized
• RedIs configuration is required only in case external backing storage is available to save evicted cache entries. this allows reducing cache miss rate and mininize access to data sources.

Diginsight.SmartCache will manage cache entries synchronization across application instances by means of the SetServiceBusCompanion.
HowTo: Configure SmartCache synchronization across application instances

STEP 03: load data by means of cacheService

load your data by means of Diginsight.SmartCache cacheService

[HttpGet("getplantscached", Name = "GetPlantsCachedAsync")]
[ApiVersion(ApiVersions.V_2024_04_26.Name)]
public async Task<IEnumerable<Plant>> GetPlantsCachedAsync()
{
    using var activity = Program.ActivitySource.StartMethodActivity(logger);

    // defines a key for the cache entry
    // NB. the cache key should include all imput parameters (that may cause different responses)
    // in this case the key is defined as a record including all relevant input parameters
    var cacheKey = new MethodCallCacheKey(cacheKeyService, 
                       typeof(PlantsController), nameof(GetPlantsCachedAsync));

    // data with max-age 10 minutes is requested
    var options = new SmartCacheOperationOptions() { MaxAge = TimeSpan.FromSeconds(600) }; 

    // Calls GetPlantsAsync by means of smartCache service
    var plants = await smartCache.GetAsync(cacheKey,
        _ => GetPlantsAsync(), 
        options);

    activity.SetOutput(plants);
    return plants;
}

the image below show the log of the SampleWebApi GetPlantsCachedAsync method.
The first call finds a cache miss and resolves to calling the GetPlantsAsync method. the following calla find a cache miss obtaining the result in 2/3ms instead of more than 1sec (about 1 to 1000 ratio).

Reference

The following articles discuss the details of Diginsight.SmartCache use and configuration:

• HowTo: Cache data, Invalidate entries and reload cache on invalidation
  discusses how to cache calls, and add support for invalidation and reload to cached data.

• HowTo: Synchronize cache entries across application instances with ServiceBusCompanion or KubernetesCompanion.
  discusses how to configure the ServiceBusCompanion or the KubernetesCompanion to support distributed cache entries across application instances.

• HowTo: Configure SmartCache size, latencies, expiration, instances synchronization and RedIs integration.
  discusses how to configure cache size, expiration latencies and connection to external RedIs backing storage.

• HowTo: Boost application performance with age sensitive data management.
  discusses how performance of our applications can be boosted by using smartcache.

• [HowTo: Enable data preloading by means of AI assisted algorithms.md]
  (TODO): explores how to enable AI assisted preloading to improve data preloading efficiency.
  
  

For more information visit: SmartCache

Contribute

Contribute to the repository with your pull requests.

• SmartCache
• Diagnostics

License

See the LICENSE file for license rights and limitations (MIT).