One Million SKUs Import - Bulk insert\update

Vote:
 

Hello,

We have a client with about 1 million SKUs in their catalog.

The challenge we are facing is how we can import/update this huge number of products in most effiicent way.

With initial running of our custom product import tool in the development environment, it takes 1 second to insert\update 2 SKUs. This means we need about 6 days to update the catalog :)

I will be optimistic and say that it would take 1 second to update 4 SKUs on production. Still, we will need about 3 days to update the whole catalog!

Our custom tool uses IContentRepository to create/update products.

We do not use CatalogEntryDto but VariationContent.

My questions are:

  1. Is there any other approach that would save time for importing such a huge catalog?
  2. Is there anyway we can do a bulk update for the catalog using IContentRepository?

Thanks!

#208317
Edited, Oct 21, 2019 22:16
Vote:
 
#208321
Oct 22, 2019 0:43
Vote:
 

Thanks Johan for your reply!

Unfortunately, this project is on version 12.17.2.

#208322
Oct 22, 2019 0:51
Vote:
 

Now you have a reason to upgrade, or are you somehow locked to that specific version?

#208323
Oct 22, 2019 1:31
Vote:
 

Agree with Johan. You have two choices, either use the CatalogImportExport which has unparellel import performance, or upgrading to 13.9 and use the new API. There is no other in-between options 

#208327
Oct 22, 2019 8:11
Vote:
 

Thank you both - I wanted to make sure there is no other alternatives.

#208365
Oct 22, 2019 15:37
This topic was created over six months ago and has been resolved. If you have a similar question, please create a new topic and refer to this one.
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.