Maybe I'm reaching here, but I thought I'd try the DDS for storing large amounts of simple POCO:s - and we're talking LARGE amounts, like 100k objects. Doesn't seem to be working out that good, can't even save 10k objects in a reasonable amount of time. Is there a way to batch save objects? Or am I just pushing the limit of the DDS' capacity? I'm leaning towards just serializing everything directly instead..
Just out of curiosity: Are you doing 10K calls to save objects or do you have an object graph containing that many objects (like an object containing a large list or something)?
I don't know all the inner workings of the DDS but I'm thinking that if you do separate calls it will end up doing separate DB calls, while when saving a large object graph at once the DDS would be "smart" and group objects into fewer inserts/updates?
If you havent tried putting your objects in an object containing a list (as you probably know you can't store a list directly), it would be interesting to know if it affects the performance.
The same question goes for using transactions - could that make performance better (or worse?). Paul describes DDS transactions here: http://world.episerver.com/Blogs/Paul-Smith/Dates1/2011/4/Dynamic-Data-Store--Tips-Tricks-and-Best-Practises/
Thanks it's a good idea, I'll try it today. I thought about transactions too but if I'm still just saving them one by one I don't think they will have an positive impact on performance (maybe worsen it). But I'll try that as well (I should be using them anyway).
Unfortunately I rembered incorrectly - I am in fact using an object graph already. 1 object that holds a few properties and a list of 10k+ simple POCO:s. I tried DDS ExecuteTransaction as well, none of these methods gave any increase in performance.
Saving 1 wrapper object that holds a list of 1k objects takes around 11 seconds. 100 objects clocks in at around 1 second.
FYI I ended up using a wrapper with a byte array for each type containing the serialized objects, then storing the wrapper to the DDS (= lightning fast). Kind of meaningless to use the DDS for this maybe but at least I don't have to write to the db myself.
Thanks for sharing your results!