Automatic 'Load More' action

Forgive me if this has already been discussed, but I feel that it is a fairly major issue. I am running into quite a few situations where I am running into the ‘Apex heap size too large’ error with some of my models; judging by some quick searching it looks like this is a fairly common issue. I know how to work around it by limiting the number of records that are returned by a single query, but I would like to hear from the Skuid developers about whether there are any better solutions for the problem (I know aggregate models can be a solution, but they have their own issues I’ll address below). I have one model (Transactions, Quarter to Date) that is particularly problematic. Early in a quarter it works just fine, but of course as the days and weeks go by the number of transactions add up and all of the sudden I am left with a Heap Size error, which I can fix by limiting the query results on the model. But that causes a host of other problems-- all of the sudden my CEO wants to know why the chart on the model page isn’t showing recent transactions, and I need to explain to him that he needs to go to a separate tab and hit the ‘Load More’ button a few times to get all of the data. That is less than ideal, to put it lightly. Would it be possible to do sequential and automated ‘Load More’ actions so that I don’t have to explain to users that they need to keep hitting ‘Load More’ in order to get their graphs and charts to reflect what is actually in the database? If I’m clicking ‘Load More’ on my own, why can’t Skuid just do it on its own a few times until the table/chart shows all of the available data? Can a snippet do this for me somehow? I have some really wonderful Skuid pages showing transactions, commissions, and other data for my users, but as the year progresses and the amount of YTD data piles up the pages are becoming increasingly difficult to manage. What solutions are the Skuid developers considering here? Instructing everyone to get in the habit of clicking ‘Load More’ every time they open a Skuid page is not a good solution, and this kind of data isn’t easily aggregated either (and aggregations bring their own set of issues with them, such as very ugly chart series names when dealing with dates).

Hi,

Have you read through this tutorial?  http://help.skuidify.com/m/11720/l/205082-table-component-custom-global-actions


There is some really good tidbits on working with the entire dataset e.g. a model with more than 10K records.

There is also sample code for paging in JavaScript.

Regards,
Irvin

I’m not sure about the details, but in theory you should be able to set your model to not load anything on pageload, and then write some javascript on pageload that would sequentially load more rows until all the rows were loaded.

Use canRetrieveMoreRows and .loadNextOffsetPage() (see skuid.model.Model)

In principle, it’s something like this:

var model = skuid&#46;$M('YourModel');<br />$(document&#46;body)&#46;one('pageload, function(){<br />&nbsp; while (model&#46;canRetrieveMoreRows) {<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;model&#46;loadNextOffsetPage();<br />}<br />});

But you're going to run into async issues. You need to tell the script to wait until the query completes before looping again. I'm not confident enough to with jQuery .when() or .deferred() to take a pass at that, but I'm sure someone is!

Yes!  What Irvin says… 

Has Anyone ever figured out the full JS to be able to accomplish this?

If yes, I would appreciate it if you could share it, as I’m having the same issue

Thank you

The trick is actually to use a self made loop and the callback on the loadNextOffsetPage function:

var loopLoad = function(){ model.loadNextOffsetPage(function(){ if(model.canRetrieveMoreRows){ loopLoad(); }else{ //do something fun with your data here if you want } }); }<br>

There is a Model prototype method that handles this looping for you: model.loadAllRemainingRecords() . Calling this single line from JavaScript will kick off logic that is functionally equivalent to what Jacob posted, with some extra capabilities in terms of callback functions.

As the docs for this method describe, calling this method will kick off a loop that will not terminate until all remaining records in the database that meet the Model’s Conditions that have not already been retrieved into the Model have been retrieved. 

NOTE: in Salesforce, there are hard limits on “offsets” which may prevent you from retrieving “all” records in your database. For custom objects and the most commonly-used standard objects, the limit is 10000 + your Model’s “Max # of Records to Retrieve (Limit)” property. For other standard objects and system objects, the limit is 2000 + “Max # of Records to Retrieve (Limit)”. 

this is AWESOME.

Agreed, this is awesome.  I was able to implement a “lightning-like” functionality of loading on scroll using this.  So when the “load more” button comes into view, it then runs the loadNextOffsetPage() to load the next batch.  Significantly improved performance!

Paul, that sounds cool! I’m curious about your solution, would be great if you could share your script here :slight_smile:

Sure, it’s a hacky prototype right now, but here it is in all of it’s raw prototype glory.

IMHO, this should be a feature in skuid without js…

Also, in full transparency for credit, the js pattern for determining if an element is in a viewport is from here:
https://medium.com/talk-like/detecting-if-an-element-is-in-the-viewport-jquery-a6a4405a3ea2

So this is a mashup of that thread and this thread…

(function(skuid){
var $ = skuid.$;
      
$.fn.isInViewport = function() {
var elementTop = $(this).offset().top; var elementBottom = elementTop + $(this).outerHeight();
var viewportTop = $(window).scrollTop();
var viewportBottom = viewportTop + $(window).height();
return elementBottom > viewportTop && elementTop < viewportBottom;
};
$(window).scroll(function() {  
          
 $('.nx-list-loadmore').each(function() {
var model = skuid.$M('');
    if ($(this).isInViewport()) {
      $.blockUI({
              message: 'Loading More Rows...',
              timeout: 1500
           });

         model.loadNextOffsetPage();
    }
  });
});
})(skuid);```
```

Thank you for sharing, Paul!

Paul,

I’m not having luck implementing this.

It looks like the only piece of this code that would need to be customized is where you have …or am I missing something else that needs to be customized?

Is there any other considerations to implementing this (e.g. table needs a maximum scrolling height and/or should there be max # records query limit set on the model)?

Thanks,

Conlan

Yes, a couple things:
- you need the table to render with the load more button, which means you need a max # of records.  It will load initially, and on each scroll, that # of records set there will load.  Something like 25 or 40 seems like a reasonable number.
- make sure it is implemented as an in-line, not a snippet or other type of resource location.

Let me know if that fixes it.

Got it working, this is great!

I didn’t have it set up as “In-Line” resource…I had it as In-Line Snippet.

Also, a couple additional things I had to set under the Table’s Display settings:

  • Pagination --> Visible Rows: Show All
  • Scrolling --> Allow Scroll Bars: unchecked 
Much appreciated!