Automatic 'Load More' action

  • 1
  • Problem
  • Updated 4 weeks ago
  • Solved
Forgive me if this has already been discussed, but I feel that it is a fairly major issue.

I am running into quite a few situations where I am running into the 'Apex heap size too large' error with some of my models; judging by some quick searching it looks like this is a fairly common issue. I know how to work around it by limiting the number of records that are returned by a single query, but I would like to hear from the Skuid developers about whether there are any better solutions for the problem (I know aggregate models can be a solution, but they have their own issues I'll address below).

I have one model (Transactions, Quarter to Date) that is particularly problematic. Early in a quarter it works just fine, but of course as the days and weeks go by the number of transactions add up and all of the sudden I am left with a Heap Size error, which I can fix by limiting the query results on the model. But that causes a host of other problems-- all of the sudden my CEO wants to know why the chart on the model page isn't showing recent transactions, and I need to explain to him that he needs to go to a separate tab and hit the 'Load More' button a few times to get all of the data. That is less than ideal, to put it lightly.

Would it be possible to do sequential and automated 'Load More' actions so that I don't have to explain to users that they need to keep hitting 'Load More' in order to get their graphs and charts to reflect what is actually in the database? If I'm clicking 'Load More' on my own, why can't Skuid just do it on its own a few times until the table/chart shows all of the available data? Can a snippet do this for me somehow?

I have some really wonderful Skuid pages showing transactions, commissions, and other data for my users, but as the year progresses and the amount of YTD data piles up the pages are becoming increasingly difficult to manage. What solutions are the Skuid developers considering here? Instructing everyone to get in the habit of clicking 'Load More' every time they open a Skuid page is not a good solution, and this kind of data isn't easily aggregated either (and aggregations bring their own set of issues with them, such as very ugly chart series names when dealing with dates).
Photo of pFranzen

pFranzen

  • 222 Points 100 badge 2x thumb
  • Frustrated

Posted 3 years ago

  • 1
Photo of Irvin Waldman

Irvin Waldman, Champion

  • 9,006 Points 5k badge 2x thumb
Hi,

Have you read through this tutorial?  http://help.skuidify.com/m/11720/l/205082-table-component-custom-global-actions


There is some really good tidbits on working with the entire dataset e.g. a model with more than 10K records.

There is also sample code for paging in JavaScript.

Regards,
Irvin
Photo of Matt Sones

Matt Sones, Champion

  • 31,478 Points 20k badge 2x thumb
I'm not sure about the details, but in theory you should be able to set your model to not load anything on pageload, and then write some javascript on pageload that would sequentially load more rows until all the rows were loaded.

Use canRetrieveMoreRows and .loadNextOffsetPage() (see skuid.model.Model)

In principle, it's something like this:
var model = skuid.$M('YourModel');
$(document.body).one('pageload, function(){
  while (model.canRetrieveMoreRows) {
                   model.loadNextOffsetPage();
}
});

But you're going to run into async issues. You need to tell the script to wait until the query completes before looping again. I'm not confident enough to with jQuery .when() or .deferred() to take a pass at that, but I'm sure someone is!

(Edited)
Photo of Rob Hatch

Rob Hatch, Official Rep

  • 44,006 Points 20k badge 2x thumb
Yes!  What Irvin says... 
Photo of Dave

Dave

  • 5,538 Points 5k badge 2x thumb
Has Anyone ever figured out the full JS to be able to accomplish this?

If yes, I would appreciate it if you could share it, as I'm having the same issue

Thank you
Photo of Jacob Cohen

Jacob Cohen

  • 60 Points
The trick is actually to use a self made loop and the callback on the loadNextOffsetPage function:

    var loopLoad = function(){
        model.loadNextOffsetPage(function(){
            if(model.canRetrieveMoreRows){
                loopLoad();
            }else{
                //do something fun with your data here if you want
            }
        }); 
    }
Photo of Zach McElrath

Zach McElrath, Employee

  • 49,056 Points 20k badge 2x thumb
There is a Model prototype method that handles this looping for you: model.loadAllRemainingRecords() . Calling this single line from JavaScript will kick off logic that is functionally equivalent to what Jacob posted, with some extra capabilities in terms of callback functions.

As the docs for this method describe, calling this method will kick off a loop that will not terminate until all remaining records in the database that meet the Model's Conditions that have not already been retrieved into the Model have been retrieved. 

NOTE: in Salesforce, there are hard limits on "offsets" which may prevent you from retrieving "all" records in your database. For custom objects and the most commonly-used standard objects, the limit is 10000 + your Model's "Max # of Records to Retrieve (Limit)" property. For other standard objects and system objects, the limit is 2000 + "Max # of Records to Retrieve (Limit)". 
Photo of Matt Sones

Matt Sones, Champion

  • 31,478 Points 20k badge 2x thumb
this is AWESOME.
Photo of Paul

Paul

  • 442 Points 250 badge 2x thumb
Agreed, this is awesome.  I was able to implement a "lightning-like" functionality of loading on scroll using this.  So when the "load more" button comes into view, it then runs the loadNextOffsetPage() to load the next batch.  Significantly improved performance!
Photo of Luzie Baumgart

Luzie Baumgart, Official Rep

  • 1,310 Points 1k badge 2x thumb
Paul, that sounds cool! I'm curious about your solution, would be great if you could share your script here :-)
Photo of Paul

Paul

  • 442 Points 250 badge 2x thumb
Sure, it's a hacky prototype right now, but here it is in all of it's raw prototype glory.

IMHO, this should be a feature in skuid without js...

Also, in full transparency for credit, the js pattern for determining if an element is in a viewport is from here:
https://medium.com/talk-like/detecting-if-an-element-is-in-the-viewport-jquery-a6a4405a3ea2

So this is a mashup of that thread and this thread...
(function(skuid){
var $ = skuid.$;
      
$.fn.isInViewport = function() {
var elementTop = $(this).offset().top; var elementBottom = elementTop + $(this).outerHeight();
var viewportTop = $(window).scrollTop();
var viewportBottom = viewportTop + $(window).height();
return elementBottom > viewportTop && elementTop < viewportBottom;
};
$(window).scroll(function() {  
          
 $('.nx-list-loadmore').each(function() {
var model = skuid.$M('<yourmodelhere>');
    if ($(this).isInViewport()) {
      $.blockUI({
              message: 'Loading More Rows...',
              timeout: 1500
           });

         model.loadNextOffsetPage();
    }
  });
});
})(skuid);



Photo of Luzie Baumgart

Luzie Baumgart, Official Rep

  • 1,310 Points 1k badge 2x thumb
Thank you for sharing, Paul!
Photo of Conlan O'Rourke

Conlan O'Rourke

  • 3,270 Points 3k badge 2x thumb
Paul,

I'm not having luck implementing this. 

It looks like the only piece of this code that would need to be customized is where you have <yourmodelhere>...or am I missing something else that needs to be customized?

Is there any other considerations to implementing this (e.g. table needs a maximum scrolling height and/or should there be max # records query limit set on the model)?


Thanks,

Conlan
Photo of Paul

Paul

  • 442 Points 250 badge 2x thumb
Yes, a couple things:
- you need the table to render with the load more button, which means you need a max # of records.  It will load initially, and on each scroll, that # of records set there will load.  Something like 25 or 40 seems like a reasonable number.
- make sure it is implemented as an in-line, not a snippet or other type of resource location.

Let me know if that fixes it.
(Edited)
Photo of Conlan O'Rourke

Conlan O'Rourke

  • 3,270 Points 3k badge 2x thumb
Got it working, this is great!

I didn't have it set up as "In-Line" resource...I had it as In-Line Snippet.

Also, a couple additional things I had to set under the Table's Display settings:
  • Pagination --> Visible Rows: Show All
  • Scrolling --> Allow Scroll Bars: unchecked 
Much appreciated!