"An item with the same key has already been added" in contentLoader.GetChildren<T>(contentLink)

Vote:
 

Epi version: 11.11.3.0
Commerce version: 12.17.2.0

We have some catalog categories that are throwing "An item with the same key has already been added" exceptions when we attempt to GetChildren on the category's contentLink. The categories in question work for the en-us culture, but the en-ca versions are blowing up. The <T> in question is our product interface. Our categories can have subcategories and the retrieval of those children works, but the product children causes that Epi call to fail.

Clearly, something is corrupt/misconfigured about those products, but I'm trying to figure out what it is (which will hopefully help me track down how it happened and prevent it in the future). Even if I could just know what key is duplicated, it might be helpful.

Partial stack trace (for some reason, the full one won't submit):

[ArgumentException: An item with the same key has already been added.]
   System.ThrowHelper.ThrowArgumentException(ExceptionResource resource) +56
   System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add) +12948432
   System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value) +16
   Mediachase.MetaDataPlus.Configurator.MetaObjectDB.LoadBatchOfMetaObjectData(MetaDataContext context, CatalogMetaObjectLoadSet loadSet) +1269
   Mediachase.MetaDataPlus.Internal.<DoLoadFromDb>d__14.MoveNext() +294
   System.Collections.Generic.List`1..ctor(IEnumerable`1 collection) +186
   System.Linq.Enumerable.ToList(IEnumerable`1 source) +54
   Mediachase.MetaDataPlus.Internal.<DoLoad>d__11.MoveNext() +703
   System.Linq.Enumerable.ToDictionary(IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer) +157
   EPiServer.Commerce.Catalog.Provider.Construction.MetaDataContentBuilder.GetMetaObjects(IDictionary`2 metaObjectIdsByClassId, String language) +334
   EPiServer.Commerce.Catalog.Provider.Construction.EntryBuilder.GetMetaObjects(IEnumerable`1 entryRows, String language) +317
   EPiServer.Commerce.Catalog.Provider.Construction.EntryBuilder.ConstructEntries(CatalogEntryDto entryDto, IDictionary`2 versionsForUnpublishedContent, IList`1 entryNodeRelations, String language) +509
   EPiServer.Commerce.Catalog.Provider.Construction.EntryBuilder.Create(IList`1 contentLinks, String language) +638
   EPiServer.Commerce.Catalog.Provider.<>c__DisplayClass10_0.<LoadSpecificContentInstances>b__0(ICatalogContentBuilder builder, IList`1 links) +16
   EPiServer.Commerce.Catalog.Provider.CatalogContentLoader.ConstructContent(IList`1 contentLinks, Func`3 createContentFunc) +339
   EPiServer.Commerce.Catalog.Provider.CatalogContentLoader.LoadSpecificContentInstances(IList`1 contentLinks, String language) +453
   EPiServer.Commerce.Catalog.Provider.<>c__DisplayClass29_0.<LoadContents>b__1(IList`1 refs) +22
   EPiServer.Commerce.Catalog.Provider.CatalogContentProvider.BatchLoad(IList`1 contentLinks, Func`2 dbLoader) +141
   EPiServer.Commerce.Catalog.Provider.CatalogContentProvider.LoadContents(IList`1 contentReferences, ILanguageSelector selector) +490
   EPiServer.Core.ContentProvider.GetContentBatch(IList`1 batch, ILanguageSelector selector, List`1& contents, Dictionary`2& contentMap) +85
   EPiServer.Core.ContentProvider.GetScatteredContents(IEnumerable`1 contentLinks, ILanguageSelector selector) +676
   EPiServer.Core.ContentProvider.LoadBatched(IList`1 contentReferences, ILanguageSelector selector) +61
   EPiServer.Core.Internal.ProviderPipelineImplementation.GetItems(ContentProvider provider, IList`1 contentLinks, LoaderOptions loaderOptions) +225
   EPiServer.Core.Internal.DefaultContentLoader.GetChildren(ContentReference contentLink, LoaderOptions loaderOptions, Int32 startIndex, Int32 maxRows) +1170
   EPiServer.Core.Internal.DefaultContentLoader.GetChildren(ContentReference contentLink, LoaderOptions loaderOptions) +67
   EPiServer.Core.Internal.DefaultContentLoader.GetChildren(ContentReference contentLink) +69
   EPiServer.Core.ContentReferenceExtension.GetChildren(ContentReference contentLink) in C:\dev\pixcms\src\Polaris.CMS.Common\Extensions\ContentReferenceExtension.cs:157

                    
#209479
Edited, Nov 14, 2019 21:51
Vote:
 

Can you share code. Specially the extension method around line 157 in ContentReferenceExtension.cs (last line in your stack trace).

#209484
Nov 15, 2019 3:02
Vote:
 

Have you run sql profiler and catched the stored procedure call for "CatalogContentProperty_LoadBatch"?

If you run that call against the database, what result do you get?

#209507
Nov 15, 2019 14:20
Vote:
 

Line 157 is:

var items = contentLoader.GetChildren<T>(contentLink).ToList();

#209513
Nov 15, 2019 15:18
Vote:
 

What we found last night and early this morning indicates that we somehow have meta fields that have culture-specific values, but they're marked as not culture-specific, so when the meta data is gathered together, Epi attempts to add multiple values for the same field to the gathered object. Still digging into how many products are in this state and how they may have gotten that way.

#209516
Nov 15, 2019 15:33
Vote:
 

It is possible that your CatalogContentProperty have duplicated rows with same ObjectId, ObjectTypeId, LanguageName and MetaFieldName

This should return those duplicated rows:

select objectid, objecttypeid, languagename, metafieldname
from CatalogContentProperty
group by objectid, objecttypeid, languagename, metafieldname
having count(*) > 1

You are of course welcome to contact developer support service for further assistance

#209520
Nov 15, 2019 16:23
Vote:
 

Quan, that's what we thought might be the problem too, but there aren't any actual duplicates of that kind.

I did also put in a ticket with support, but posted here because sometimes other users have found problems and solutions that the official support hasn't seen.

#209522
Nov 15, 2019 16:54
Vote:
 

Quan, that's what we thought might be the problem too, but there aren't any actual duplicates of that kind.

I did also put in a ticket with support, but posted here because sometimes other users have found problems and solutions that the official support hasn't seen.

#209523
Nov 15, 2019 16:54
Vote:
 

It's good that you contacted developer support service. We might need to look into your database to see what is in there ..

#209551
Nov 18, 2019 9:59
Vote:
 

I also had this issue and ran this query to identify mismatches between culture specific metafields and not culture specific property values for those metafields:

SELECT
	[ObjectId],
	[MetaFieldName],
	CCP.[MetaFieldId],
	[LanguageName],
	[CultureSpecific],
	MF.[MultiLanguageValue]
FROM CatalogContentProperty CCP
INNER JOIN MetaField MF ON MF.MetaFieldId = CCP.MetaFieldId
where [CultureSpecific] = 0
and MF.[MultiLanguageValue] = 1

The solution was to run this query to fix the issue:

UPDATE CatalogContentProperty
SET [CultureSpecific] = 1
WHERE [CultureSpecific] = 0
AND [MetaFieldId] IN (
	SELECT DISTINCT
		CCP.[MetaFieldId]
	FROM CatalogContentProperty CCP
	INNER JOIN MetaField MF ON MF.MetaFieldId = CCP.MetaFieldId
	WHERE [CultureSpecific] = 0
	AND MF.[MultiLanguageValue] = 1
)
#216441
Feb 06, 2020 11:00
Quan Mai - Feb 06, 2020 11:08
UPDATE CatalogContentProperty
SET [CultureSpecific] = 1
WHERE [CultureSpecific] = 0
AND [MetaFieldId] IN (
SELECT MetaFieldId
FROM
MetaField MF
WHERE MF.[MultiLanguageValue] = 1
)

it can be simplified. I don't know why you have the mismatches however
Mattias Olsson - Feb 06, 2020 11:14
It doesn't have to be simplified. I run it once, done. ;) But yeah, I agree that your script is better. :)
I don't know about the mismatches either.
Mattias Olsson - Feb 06, 2020 13:12
@Quan Mai: What if a metafield, at some point, is changed from non multilanguage to multilanguage, will data in CatalogContentProperty auto heal then?
Quan Mai - Feb 06, 2020 13:14
If you do that by the API, that should be taken care of
Mattias Olsson - Feb 06, 2020 13:17
In our case, we use InRiver. So all data is basically in a catalog XML file and sent to the importer.
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.