I’m running into an issue with the canRetrieveMoreRows and loadNextOffsetPage() properties of the model object. They seem to give inaccurate/non-consistent results when the data set has 10000+ rows.
canRetrieveMoreRows seems to return false if the next “page” of data would push the total # of records over 10000. For example, if I run the model.soql query property I get back 10000 rows -but canRetrieveMoreRows is false when the model has only 9990 rows.
Additionally, the max # of rows seems to vary depending on the page size. If I use a page size of 2000, I’ll max out at 9990 records. However, a page size of 1500 will max at 8501 records. In both situations, there should be 12201 total records.
Thanks for the help!
It looks like there are 2 issues here and I’ll try to explain them both.
1. The first issue is that you’re not getting all 12201 records. Under the hood, Skuid is using Salesforce’s StandardSetController to handle loading data when you use loadNextOffsetPage(). As you can see here, they only support loading in up to 10,000 rows.
2. The second issue is the inconsistent results. I’ve seen this before and it seems to be a quirk with Salesforce’s StandardSetController. When Skuid paginates through the StandardSetController, sometimes the StandardSetController returns rows that it had already returned on the previous page. Skuid models are smart enough to not let 2 records with the same id into them, but that’s what is causing the less than expected numbers for your models. One way that I’ve found to work around this issue is to put a good explicit order by on your model. If your model had no order by originally, order by something like “SystemModStamp, Id”. If your model already had an order by on it, try adding “Id” to the end of your order by statement. I have found this to fix the issue.
The second issue turned out to have two underlying problems. The first was the one you mentioned, and adding Id at the end of the order by seems to resolve that issue.
The other part was that the code I wrote increased the recordsLimit to 2000 (from 250) before loading the next offset page. I expected that that the StandardSetController would return records 251-2250 on the first load, when it actually loaded in rows 2001-4000. I altered my code to run updateData() once before loading in any of the offset pages and that resolved the problem.
Thanks for the help!
Glad to know you got this worked out…
I do not understand the “second issue” reported and resolved here. Running skuid 11.2.11 in early 2019 we found a model retrieving 20 rows on initial load, ordering by a custom date field (Note_Date__c). These settings are defined in the skuid page. The last record displayed on the page is from 1/4/2019. When we load 20 more records, the next record is also from 1/4/2019.
There should be a third record from 1/4/2019. It is not there.
The page has a date-range filter on Note_Date__c. If we set the filter to 1/3/2019 through 1/5/2019 – and this is the only change – All three of the 1/4/2019 records are displayed!
Furthermore, if I go to the developer tools in the browser and grab the soql for the unfiltered model (skuid.$M(“Notes”).soql), and run the soql in the Salesforce developer console, 29 rows are retrieved (including three dated 1/4/2019.) With the browser developer tools, skuid.$M(“Notes”).data.length reads 28.
I did add Id to the order by setting on the model, and the problem went away.