Mark, I’m impressed you’ve been able to make this work but I’d like to understand more about why you needed to write this. You mentioned this as the motivation:
“to prevent heap size errors from salesforce from having too large queries (sort of like load all remaining records, but without the 10,000 record limit and works on aggregate models)”
In my mind you really shouldn’t need to write a function like this with Skuid, this is what Skuid (ideally) exists to prevent you from needing to do — but it seems like you’re running into two issues:
(1) you can’t “load more” than 10,000 remaining records
(2) the “load all remaining records” approach doesn’t work for Aggregate Model load mores
Is that correct?
To address this, your solution is to:
(1) for non-aggregate models — use an Id-ordering based approach to ensure that you’re only loading additional records you haven’t loaded yet.
(2) for aggregate models — you’re using the “ORDER BY” field on the aggregate model, or examining the grouping fields, to ensure that you are only querying records that you haven’t seen yet.
I’m bummed that you are having to write this code in order to work around these problems — we really should address these issues in the product to prevent you from having to do this.
One big concern I have with building this query client-side and using skuid.sfdc.api.query() is that you’re avoiding all of Skuid’s object-level and field-level security enforcement logic which is performed server-side as part of Skuid Model loads and our Apex logic.
But my bigger concern is that you’re having to write your own query builder logic to work around a Skuid limitation.
I’m going to raise this concern with our product management team — I’ll try to post back here when I get an update.
