Another Fun and Profit with IMemory caching

Hi all,

As you might know Quan Mai already had a great post: https://world.episerver.com/blogs/Quan-Mai/Dates/2019/12/use-memorycache-in-your-project---for-fun-and-profit/

Today I want to introduce other way base on 

Microsoft.Extensions.Caching.Memory

Some references for you:

1. https://www.nuget.org/packages/Microsoft.Extensions.Caching.Memory/

2. https://docs.microsoft.com/en-us/dotnet/api/microsoft.extensions.caching.memory.memorycache?view=dotnet-plat-ext-3.1

If you make a search with "IMemory" then quite a lot of tutorial but almost is in .Net Core but doesn't blocking us to apply on .NET and EPiServer as well!

Okay, lets go step by step!

1. Create your interceptor class: MemoryCacheInterceptor
2. Intercep via ConfigurationModule and StructureMap interceptors pattern: MemoryCacheConfigurationModule

[InitializableModule]
[ModuleDependency(typeof(FrameworkInitialization))]
public class MemoryCacheConfigurationModule : IConfigurableModule, IInitializableModule
{
    public void ConfigureContainer(ServiceConfigurationContext context)
    {
        context.ConfigurationComplete += (o, e) =>
        {
            e.Services.Intercept<IObjectInstanceCache>((locator, httpRuntimeCache) =>
            {
                return new MemoryCacheInterceptor();
            });
        };
    }

    public void Initialize(InitializationEngine context)
    {
    }

    public void Uninitialize(InitializationEngine context)
    {
    }
}

/// <summary>
/// Implement IObjectInstanceCache with IMemoryCache
/// <see cref="Microsoft.Extensions.Caching.Memory.IMemoryCache"/>
/// </summary>
public class MemoryCacheInterceptor : IObjectInstanceCache, IDisposable
{
    // Interesting things will come here
}

Our main focus will be MemoryCacheInterceptor class, as you see its inherited from IObjectInstanceCache and IDisposable then we should implement those methods below:

0. Properties and Constructor

private static readonly ILog _log = LogManager.GetLogger(typeof(MemoryCacheInterceptor));

private readonly IMemoryCache _memoryCache;
private readonly object _dependencyObject;
private CancellationTokenSource _rootTokenSource;

private readonly ConcurrentDictionary<string, CancellationTokenSource> _tokenSources;

public MemoryCacheInterceptor()
{
    _memoryCache = new MemoryCache(new MemoryCacheOptions());
    _dependencyObject = new object();
    _rootTokenSource = new CancellationTokenSource();
    _tokenSources = new ConcurrentDictionary<string, CancellationTokenSource>();
    _log.Info("Started NitecoMemeoryCacheInterceptor");
}

As you see:

_memoryCache : Create new instance of MemoryCache with posibility of customize option on MemoryCacheOptions like how much memory will be used ... See more in: https://docs.microsoft.com/en-us/dotnet/api/microsoft.extensions.caching.memory.memorycacheoptions?view=dotnet-plat-ext-3.1

_dependencyObject : Dummy object cache of all master keys.

_rootTokenSource that is IMemory techinique based on CancellationToken that will help us to to invalidate a set of cache all in one go. You will see relations betweeen a CacheEntry and the token later on Insert method! 

_tokenSources : Tokens provider for each cache key

1. public void Clear()

public void Clear()
{
    if (_rootTokenSource != null && !_rootTokenSource.IsCancellationRequested && _rootTokenSource.Token.CanBeCanceled)
    {
        _rootTokenSource.Cancel();
        _rootTokenSource.Dispose();
    }

    _rootTokenSource = new CancellationTokenSource();
}

2. public object Get(string key)

public object Get(string key)
{
    return _memoryCache.Get(key);
}

3. public void Remove(string key)

public void Remove(string key)
{
    _memoryCache.Remove(key);
}

4. public void Dispose()

public void Dispose()
{
    _memoryCache.Dispose();
}

5. public void Insert(string key, object value, CacheEvictionPolicy evictionPolicy)

That is main part and we should take a coffee then focusing on ;)

...

public void Insert(string key, object value, CacheEvictionPolicy evictionPolicy)
{
    if (evictionPolicy == null)
    {
        // depend on root token only
        _memoryCache.Set(key, value, new CancellationChangeToken(_rootTokenSource.Token));
        return;
    }

    // Try setup dummy master object cache.
    EnsureMasterKeys(evictionPolicy.MasterKeys);

    using (var cacheEntry = _memoryCache.CreateEntry(key))
    {
        // Propagate tokens to current cache entry. 
        AddDependencyTokens(evictionPolicy);

        var cacheEntryOption = GetCachEntryOption(key, evictionPolicy);
        cacheEntry.SetOptions(cacheEntryOption.AddExpirationToken(new CancellationChangeToken(_rootTokenSource.Token)));
        cacheEntry.SetValue(value);
    }
}

Okay are you ready?

Simple part in the function is: Check null of evictionPolicy then insert to memory but remember to tight this cache key on our root token source to eviction all cache on Clear method! Done!

if (evictionPolicy == null)
{
    // depend on root token only
    _memoryCache.Set(key, value, new CancellationChangeToken(_rootTokenSource.Token));
    return;
}

Next we ensure master keys are ready to use (have cached object) with EnsureMasterKeys(evictionPolicy.MasterKeys) method:

private void EnsureMasterKeys(string[] masterKeys)
{
    if (masterKeys != null)
    {
        foreach (string key in masterKeys)
        {
            object cached;
            if (!_memoryCache.TryGetValue(key, out cached))
            {
                var token = _tokenSources.GetOrAdd(key, new CancellationTokenSource());
                _memoryCache.Set(key, _dependencyObject,
                    new MemoryCacheEntryOptions()
                    .SetAbsoluteExpiration(DateTimeOffset.MaxValue)
                    .SetPriority(CacheItemPriority.NeverRemove)
                    .AddExpirationToken(new CancellationChangeToken(_rootTokenSource.Token))
                    .AddExpirationToken(new CancellationChangeToken(token.Token))
                    .RegisterPostEvictionCallback(PostEvictionCallback));
            }
        }
    }
}

Our idea here is: Loop through master kesy, if it's not existed then insert with an absoluted maximum time and highest cache priority so in theory it will never been automactically removed by Garbage Collection (GC). Then create  token source for it, add the token and root token source together to the cache entry option.

Oh but what and why do we need PostEvictionCallback ? and when it will be trigerred? 

What and When:The given callback will be fired after the cache entry is evicted from the cache.  Read more

Why : Whenever the master cache is evicted then we should remove all children depend on the master token!

private void PostEvictionCallback(object key, object value, EvictionReason evictionReason, object state)
{
    CancellationTokenSource token;
    if (_tokenSources.TryRemove((string)key, out token)
        && token != null && !token.IsCancellationRequested && token.Token.CanBeCanceled)
    {
        token.Cancel();
        token.Dispose();
    }
}

Last part: Create new cache entry, add dependency tokens and of course with root token as well and then set to memory!

using (var cacheEntry = _memoryCache.CreateEntry(key))
{
    // Propagate tokens to current cache entry. 
    AddDependencyTokens(evictionPolicy);

    var cacheEntryOption = GetCachEntryOption(key, evictionPolicy);
    cacheEntry.SetOptions(cacheEntryOption.AddExpirationToken(new CancellationChangeToken(_rootTokenSource.Token)));
    cacheEntry.SetValue(value);
}

The magic part AddDependencyTokens(evictionPolicy)

private void AddDependencyTokens(CacheEvictionPolicy evictionPolicy)
{
    var dependencies = evictionPolicy.CacheKeys;
    if (dependencies == null)
    {
        dependencies = evictionPolicy.MasterKeys;
    }
    else if (evictionPolicy.MasterKeys != null)
    {
        dependencies = dependencies.Concat(evictionPolicy.MasterKeys).ToArray();
    }

    if (dependencies == null) return;

    foreach (var k in dependencies)
    {
        object v;
        _memoryCache.TryGetValue(k, out v);
    }

}

Hmm, quite strange right? Firstly, combination cache keys with master keys (we're considering cache keys and master keys are dependencies) and then just do: TryGetValue ? Is that enough? Yes, because of when you use:

using (var cacheEntry = _memoryCache.CreateEntry(key))

then you put this entry on the top of scope, see more:  Microsoft.Extensions.Caching.Memory.CacheEntry.CacheEntry

internal CacheEntry(object key, Action<CacheEntry> notifyCacheEntryDisposed, Action<CacheEntry> notifyCacheOfExpiration)
{
	...
	_scope = CacheEntryHelper.EnterScope(this);
}

and  Microsoft.Extensions.Caching.Memory.MemoryCache.TryGetValue

public bool TryGetValue(object key, out object result)
{
    ...
    value.PropagateOptions(CacheEntryHelper.Current);
    ...
}

Next is simple but important function GetCachEntryOption 

private MemoryCacheEntryOptions GetCachEntryOption(string key, CacheEvictionPolicy evictionPolicy)
{
    var cacheEntryOption = new MemoryCacheEntryOptions();

    switch (evictionPolicy.TimeoutType)
    {
        case CacheTimeoutType.Undefined:
            break;
        case CacheTimeoutType.Sliding:
            cacheEntryOption = cacheEntryOption.SetSlidingExpiration(evictionPolicy.Expiration);
            break;
        case CacheTimeoutType.Absolute:
            cacheEntryOption = cacheEntryOption.SetAbsoluteExpiration(evictionPolicy.Expiration);
            break;
    }

    var tokenSource = _tokenSources.GetOrAdd(key, new CancellationTokenSource());
    return cacheEntryOption
            .AddExpirationToken(new CancellationChangeToken(tokenSource.Token))
            .RegisterPostEvictionCallback(PostEvictionCallback);
}

Once againe you see PostEvictionCallback here with the same logic: When cache is evicted then we will cancel the token so that it will evict all others dependent cache entries!

Viola! Happy Coding!