Question regarding caching

Vote:
 

Caching for each subsystem, such as catalogs and orders, is configured in their respective configuration files. For catalogs for example, see ecf.catalog.config in the configs folder of the site.

The collectionTimeout responds to an entry array, and "entry" responds to a single entry. What is actually cached is the CatalogEntryDto and, because the Entry object is created from the Data Transfer Object (DTO), you cache the DTO. 

The following is found in Commerce caching documentation: https://world.episerver.com/documentation/Items/Developers-Guide/Episerver-Commerce/9/Caching/Caching/

So, the fact that it's the CatalogEntryDto that is cached, does that mean that this cache is not relevant if you are only accessing commerce objects through ReferenceConverter and IContentLoader? Or is this cache used when browsing the Commerce Ui?

#187852
Feb 05, 2018 10:15
Vote:
 

This question is interesting.

Internally the DTOs are loaded to construct the contents, so in older versions, when you load a content, the DTO is actually cached, unless if you set that timeout to 00:00:00, which means no caching.

In newer versions, 11.0 or something, I can't remember the exact version, we added smarter cache, so if you load the content, the DTO is not cached to reduce the cache footprint. (the content itself is cached by default, so it does not make senses to cache to DTO in lower level). I think my colleague Magnus Rahl blogged about that some time ago. 

#187853
Feb 05, 2018 10:20
Vote:
 

Thanks for your fast reply, as always, Quan! 

We are running v11.4 of Commerce and have an issue were we are running out of memory when the Find indexing job runs (the site runs in Azure as S3). The catalog size is quite big with +100 000 products. 

One of the theories was that we might be caching double if catalc cahing was enabled in ecf.catalog.config, but I guess that is not the case here.

#187856
Feb 05, 2018 10:39
Vote:
 

It's more like the content cache which fills up the memory. You actually can control the cache timeout by CommerceCacheSettings.DefaultCacheTimeOut (default is 10 minutes). But this is internal setting, so use it at your own risk.

I heard CMS 11 has some more refined controls for the scheduled job so you can basically tell the system to not cache at all for the content loaded by a scheduled job, that that would require Commerce 11.5+ 

#187857
Feb 05, 2018 10:46
Vote:
 

That is interesting. Is this documented somewhere? I couldn't find it in the release notes or in the documentation.

#187860
Feb 05, 2018 11:05
Vote:
 

Either that or make sure you have +100 000 * (mean cache size for one IContent) bytes available RAM! :) But anything that loops through the entire catalog should probably try to not meddle with the cache (except using if already cached perhaps). Seeing as you likely don't have that amount of RAM you'll do a whole lot of evictions during the loop and your result cache might not be optimal (i.e. IContent that is rarely used on the site, but they were latest in the indexing job).

#187861
Feb 05, 2018 11:07
Vote:
 

When we run a scheduled job, we create an isolated empty cache for that job, then after the job is done we clear it. This to make sure that things picked up by the job more than once is done efficiently, but also that other threads/jobs don't interfere. Of course we have to only run this on a separate machine, as the RAM issue would still stand, we can't run it on a public site with visitors :) 

#187862
Feb 05, 2018 11:12
Vote:
 

@Mari if you're running out of memory it can also be because of these .NET bugs: https://world.episerver.com/blogs/Magnus-Rahl/Dates/2017/11/two-bugs-in-aspnet-that-break-cache-memory-management/

Another thing relating to the find indexing job is that from CMS 11.1, scheduled jobs use a significantly shorter cache timeout for the objects it inserts (default 1 minute, but configurable), see https://world.episerver.com/documentation/developer-guides/CMS/scheduled-jobs/ (edit: I see Quan already mentioned this)

#187934
Edited, Feb 06, 2018 19:56
Vote:
 

Thanks, @Magnus. I'll investigate if the .NET bugs might be the problem.

#187947
Feb 07, 2018 9:15
Vote:
 

What if I want to manually clear cached content, how would I construct the key to find in this call CatalogCache.Delete(key)?  

"NodeEntryRelation:123"   (123 = CatalogEntryId)

For example we run a product importer job that changes the title on a product so we'd want the cache to clear out right away.

#203753
May 06, 2019 17:08
Vote:
 

It sounds like you are using direct database access to update the product, which is simply a no no. 

#203754
May 06, 2019 17:47
Vote:
 

Oh hell no, we never do that.  We're going through the IContentRespository.  We have our own caching solution on the site and we use Epi caching in commerce.  We want to cache longer on our end, but need to invalidate epi cache on demand.

Thanks

#203757
May 06, 2019 18:22
Vote:
 

I would advice against doing that. you can lower the cache time for catalog content as above post, but you should not try to remove individual cache item 

#203775
May 07, 2019 10:35
Vote:
 

Thanks Quan.  So if I update information about a product with our importer (through IContentRepository), will that product and all information related be purged from epi cache?

#203817
May 07, 2019 21:43
Vote:
 

Yes, it should. We spent a lot of time making sure it does.

#203820
May 08, 2019 2:06