Heap size issues with geo charts

  • 1
  • Question
  • Updated 4 years ago
  • Answered
I know this has been discussed extensively on the community, but... Large data sets can be hard to work with in skuid. I have been able to work around them for the most part by getting the users to understand the "Load More" button and keeping my queries as specific as possible. However the GeoChart component is really cool and I was hoping to be able to map all of our accounts by state. We have something like 100,000 or so but I can't get past 5000 without running into heap size issues. This component seems to be built specifically for mapping large amounts of data, but if I can't get past 5000 that severely limits what I can do with it. There also doesn't seem to be a "Load More" button available for the chart components, so I really have no way of getting the data into the page. I understand that this was a late addition to this release I just think it would be so much more powerful with expanded data volume handling abilities.
Photo of Moshe Karmel

Moshe Karmel, Champion

  • 8,646 Points 5k badge 2x thumb

Posted 4 years ago

  • 1
Photo of mB Pat Vachon

mB Pat Vachon, Champion

  • 42,674 Points 20k badge 2x thumb
Can you use an aggregate model for this? I don't know as I haven't tried.
Photo of Rob Hatch

Rob Hatch, Official Rep

  • 44,006 Points 20k badge 2x thumb
Pat is on the money.  Use an aggregate query so that the processing occurs on the Heavy Iron,  rather than trying to pass all that data across the wire.   Then your model has a very manageable amount of data.  GeoCharts should work with Aggregates. 
Photo of Moshe Karmel

Moshe Karmel, Champion

  • 8,646 Points 5k badge 2x thumb
Rob do you think you can put together a tutorial or something. I'm not sure what exactly I would be aggregating.... Meaning should I do a SUM of BillingState or a COUNT, that's not going to help the chart which is expecting the actual state code. Basically I don't understand how I would set up the aggregations.
Photo of Rob Hatch

Rob Hatch, Official Rep

  • 44,006 Points 20k badge 2x thumb
Moshe,  Enclosed below is some XML:   This aggregates all your accounts by state and country and shows this on a map. 

<skuidpage unsavedchangeswarning="yes" showsidebar="true" showheader="true" tabtooverride="Account">
<models>
<model id="Account" limit="" query="true" createrowifnonefound="false" sobject="Account" doclone="" type="aggregate">
<fields>
<field id="Id" name="countId" function="COUNT"/>
</fields>
<conditions>
<condition type="join" value="" field="Id" state="" operator="in" enclosevalueinquotes="true" joinfield="AccountId" joinobject="Opportunity">
<conditions>
<condition type="fieldvalue" value="Closed Won" enclosevalueinquotes="true" field="StageName" operator="="/>
<condition type="fieldvalue" value="1" enclosevalueinquotes="false" field="IntegerSubscriptions__c" operator="gte"/>
</conditions>
</condition>
</conditions>
<actions/>
<groupby method="simple">
<field id="BillingState" name="billingState"/>
<field id="BillingCountry" name="billingCountry"/>
</groupby>
</model>
</models>
<components>
<pagetitle model="Account">
<maintitle>
<template>{{Model.labelPlural}}</template>
</maintitle>
<subtitle>
<template>Home</template>
</subtitle>
<actions>
<action type="savecancel"/>
</actions>
</pagetitle>
<skuidvis__geochart model="Account" maintitle="{{Model.labelPlural}}" maptype="world-highres" height="600px" backgroundcolor="#9bc6cf" nullcolor="#eee" rbwidth="0" dlcolor="black" dlbold="false" dlstrokewidth="0" mapgroup="global" dlformat="country-abbrev">
<colors/>
<legend/>
<serieslist>
<series joinBy="hc-key" valueField="countId" countryfield="billingCountry" statefield="billingState" aggFunction="sum"/>
</serieslist>
<buckets>
<bucket label="None" from="0" to="0" color="white"/>
<bucket from="0" to="5" label="1-5"/>
<bucket label="6-100" from="6" to="100"/>
<bucket label="101-1000" from="101" to="1000"/>
</buckets>
</skuidvis__geochart>
<skootable showconditions="true" showsavecancel="false" searchmethod="server" searchbox="true" showexportbuttons="false" pagesize="10" createrecords="false" model="Account" mode="readonly">
<fields>
<field id="BillingCountry" name="billingCountry" valuehalign="" type=""/>
<field id="BillingState" name="billingState" valuehalign="" type=""/>
<field id="Id" name="countId" valuehalign="" type=""/>
</fields>
<rowactions/>
<massactions usefirstitemasdefault="true"/>
<views>
<view type="standard"/>
</views>
</skootable>
</components>
<resources>
<labels/>
<css/>
<javascript/>
</resources>
</skuidpage>
Photo of Moshe Karmel

Moshe Karmel, Champion

  • 8,646 Points 5k badge 2x thumb
That's perfect!
Photo of Peter Baeza

Peter Baeza

  • 2,868 Points 2k badge 2x thumb
Aggregate models is a good solution. Another option for trend analysis and geocharts without having to load very large datasets would be to use Analytic Snapshots. Your Skuid model would then only query that aggregated snapshot object. Kind of like saving an aggregate model on regular intervals.
Photo of mB Pat Vachon

mB Pat Vachon, Champion

  • 42,674 Points 20k badge 2x thumb
Really like this. You could progress of territories over time.