PageInfo.PageSizePageInfo

Member since: 2016

Hi all!

I was wondering if anyone knows why

PageInfo.PageSize

has a secret upper limit that is less than it's data type max value (int32.max)?

Also why does that upper limit differ when doing queries to Comments and Ratings? I recall Ratings has a much lesser value than Comments. When going over this value you get an exception.


Regards,
Joel Yourstone

#174887 Feb 07, 2017 11:34
  • Member since: 2015

    Hey Joel,

    We encourage modest paging settings for performance reasons; larger pages or deep offsets can negatively impact query and delivery times. At an extreme, for example, 10k per content item * 1000 items produces approximately a 10mb result page. That's not a huge sum these days but it's not ideal if you need to deliver it frequently. For scenarios where you might be doing out-of-process work with larger amounts of data (e.g. custom analysis of ratings data), we recommend finding a modest limit that suits your needs and iterating over the pages of the result set. 

    The developer guide has some helpful info on these topics:

    I have also added a ticket to the backlog to investigate the inconsistency that you encountered as well as to explore where the documentation can be clarified.

    Thanks!

    #174894 Feb 07, 2017 14:42
  • Member since: 2016

    Hi Chris,

    Great! I think some clarity is all we need, right now we have some hard coded integers with comments saying we don't really know what the roof limit is, but we want to take as many as we can.

    Also I do see some use cases where fetching everything is required, for example when migrating data, or adding/removing a field. Now we've had to survive with two nested loops, one for all the entities in ResultPage.Results and one for doing the query again, with offset, if ResultPage.HasMore is true.

    I'm all aboard querying for performance, but sometimes you need to do expensive stuff, and that should be up to the developer in my view, not a limit in the API. The API documentation should be very clear with best practices, as it already is with the links you provided, however! :)

    Regards,
    Joel Yourstone

    #174897 Feb 07, 2017 15:03