Custom Metadata - A better way to manage Skuid page/pageassignment/etc. migrations?

Not sure I follow you completely, sorry.  Can you explain step-by-step what you’re thinking?

Pat,

You might want to look at Blue Canvas (https://bluecanvas.io/) as another option for simple source control of Salesforce.

If you want to trigger an external system when a page changes, you can also look at Salesforce’s Platform Events.  They just released this in Summer 17.   It will create an immutable object for each change of any object.  It publishes this change on a ‘bus’ that you can have an external application ‘subscribe to’.  This link has a nice overview:  https://developer.salesforce.com/blogs/developer-relations/2017/05/first-impressions-platform-events…

As far as where you want to go with your setup to manage Skuid page changes in source control, I think you should go down the route of creating a Static Resource per Skuid Page/Assignment/Theme.  This will make it easy to merge individual items if needed and you can granularly move items from Dev to Full to Prod.  This means you’ll need a trigger to create/save the static resource and an invocable action that you can run from a Skuid button to pull all changes from the static resource just moved into a new org.  Note that in my ‘design’ you’ll need to watch the Static Resource for changes and have them committed to Source Control.  This is something that BlueCanvas does out of the box.

Thanks,

Bill

We create a sandbox, branch and when need be a static resource for the Skuid page(s) for each feature using Jira’s issue key to name them all. The process works similarly to Bills, but it doesn’t have the automation. 

Hey Bill. Can I see your setup works? What do you use to commit changes from the sandbox to the feature branch?

Blue Canvas looks nice. Not as far along as Gearset, but the continuous monitoring and deployment functionality could make life much easier.

Anyone care to develop the triggers and invocable methods that Bill suggested? Or at least get us started?

Pat - So at which point do you create the page pack?  How many Skuid pages does your org typically have? I’m still not following the entire process that you’re using, sorry.  Can you walk it through step-by-step?  For example:

1) Feature is requested
2) Issue gets created in JIRA
3) SCRUM occurs and feature gets assigned
4) …

Bill - BlueCanvas looks intriguing, thanks for pointing this out.  It appears that it automates some of the processes that we’ve been discussing here with regards to keeping org & VCS in sync - it’s the build vs. buy decision and I’m always a big fan of “buy” vs “build” :)  On the downside, the challenge with having a static resource for every page/assignment/theme is that you could quickly end up with lots of these.  In the last Skuid app I worked on, we had hundreds of pages.  Do you have any experience with using your ‘design’ in larger orgs with lots of Skuid pages?

Matt - (Note: Friday humor coming in 3…2…1) Probably not what you were thinking when you asked to ‘get us started’ but  here’s a start :slight_smile:

trigger trgPageToVCS on skuid__Page__c (after insert, after update, after delete, after undelete) {<br />&nbsp; &nbsp;&#47;&#47; create static resource for page<br />&nbsp; &#47;&#47;&nbsp; or publish event/make outbound call to 'notify' background process to grab latest page<br />}

Brilliant, @Barry! That’s exactly what I needed! :wink:

On a serious note… what 's wrong with having lots of static resources?

Glad you enjoyed Matt :) 

Technically, there’s nothing wrong with lots of static resources per se.  However, the more “stuff” your managing via non-native processes the more risk you introduce in to your overall solution.  Additionally, org refreshes with dev tools, etc. become slower as do deployments (more files to sync) thereby decreasing developer efficiency.  Lastly, there are governors on static resources (no file larger than 5MB & 250MB total).  Technically it can work, it just introduces variables and in-turn risk that would otherwise be unnecessary.

I do understand and appreciate your guys desire to find declarative solutions to manage Skuid Pages with a VCS and there are some solutions that will work.  The main challenge is that there isn’t an available tool today that does exactly what you guys want/need and therefore a custom solution is required to facilitate it.  While this can be done, that solution needs to be maintained & supported which takes away from focusing on running your business and adding value to that business.  Personally, I just think that the increased “risk” and “overhead” associated with any of those processes isn’t worth the “reward.”  That said, happy to help continue to find possible solutions to the problem - i just wouldn’t recommend any of them :slight_smile:

The idea of just notifying the VCS (or some kind of system) to run ‘skuid-pull’ for the changed page seems ideal to me.  Are there servics that could do that kind of thing?

I’m completely out of my depth here. I’ve picked up skuid-grunt and use the CLI for skuid and git with bitbucket (turns out the commands are much faster than using SourceTree).

We’ve been pounding away at trying to automate some kind of deployment system that would work well for both salesforce metadata and skuid data.

Let me try to come up with a fancy picture to demonstrate what our system looks like, and then maybe you folks can offer some suggestions?

That would be very helpful Matt.  The more details on what you guys currently have process & tools wise the more meaningful our discussions around them can be.

Regarding just notifying the VCS to run the CLI commands, if it were me and I was absolutely forced to find a declarative solution, I would target the absolute minimum to get the job done.  This would mean, as you suggest, finding a simple way to initiate a “skuid-pull/push/etc.” when a page is changed.  In my other post, I mentioned creating a GUI wrapper on top of the CLI tools or a browser extension - either of these would probably be where I’d start.  This avoids having to do anything on the SFDC side (e.g. triggers, events, etc.) and instead, just requires the developer to remember to push the button in the GUI when they are done developing their page.  The GUI could use SFDC apis and SOQL to query for information, pages, etc. and present the user with a drop down that allows them to “pick” which page they want to “commit to VCS”. The browser extension could just take the current page information and initiate the CLI commands although this gets a little complicated due to security limitations from within a browser.

Ok, here’s approximately what we’re doing at the moment…
It’s a manually intensive process, which is mostly working, but we run into hiccups primarily with merging SFDC metadata between orgs after a git merge.

We were attempting to use grunt to create an automated deployment process–basically looping through all our customers and running all the commands that we run on the command line in the right sequence–but it’s fraught with opportunities for failure.

Suggestions?

We don’t necessarily need a clicks-only solution (although it would make it easier for the over Dev on my team). I’m mostly interested in finding a way to better streamline or automate the deployment process.

Barry,

Nice start…where’s the Github/Gitlab/Bitbucket repo???  :slight_smile:

Unfortunately, I don’t have experience using my proposed design.  I was ‘threading’ off a Pat’s design and adding some options to the mix.  I have used Skuid Grunt, but primarily for page backup and check in to Git.  I have had the same thoughts as Pat in that there ‘ought to be a better way’.  I just haven’t pursued them in earnest.

I think BlueCanvas has the nicest out of box experience for declarative and Apex developers.  It keeps track of everything with limited fuss.

Thanks,

Bill

So many ways to do this! I already know what Skuid likely does with their own org since they’ve got engineers up the wazoo.

Assuming no failed tests, our process is as follows.

  1. Start new feature.
    a) Create Page Pack with needed pages named as per Jira issue key.
    b) Create new sandbox named as per Jira issue key.
    c) Create branch from Jira off master named as per Jira issue key. (BitBucket integration in Jira)
  2. Develop
    a) On dev sandbox
  3. Commit to Branch
    a) Update page pack
    a) Compare and Deploy from Sandbox to Branch using Gearset
  4. Pull Request
    a) Create Pull Request from Jira. (BitBucket integration in Jira)
  5. Code Review and Merge
    a) In BitBucket
  6. Continuous Integration
    a) Auto Deployed to partial sandbox for apex tests.
  7. Deploy to UAT
    a) Deploy to Full Sandbox using Gearset.
  8. Deploy to Production

Workflow in Jira

Diagram of the process

Hey Matt -

Appreciate the pictures, helpful to better understand, thank you!  The one thing I learned from working with SFDC is that managing “source” is extremely challenging and ripe for failure.  You do need to be meticulous in the way that you manage things and unfortunately, the tools available today just aren’t what they need to be.  The introduction of skuid-grunt was very helpful and moved the needle forward but challenges still are present with SFDC metadata and its limitations as well as skuid resources.  In short, as much as SFDC wants everyone to believe that it’s an “enterprise development” platform, in my opinion it’s not.  The tooling just isn’t there - although it’s come a long way in the last year (e.g. VS Code extension, SFDC DX, etc.) and hopefully will continue to get better.

Questions:
1) Can you provide some specifics on what challenges you’re having with merging metadata?  One thing I’ve noticed in the past (although it’s been quite a while) is that if you don’t keep your sandboxes in sync then merges become much more difficult especially around “SFDC Upgrade” time when new features/metadata gets introduced to the orgs.

2) For the automation you do/did have in place, looping through customers, what challenges or opportunities for failure were you facing?


Thanks Bill, all makes sense and agreed - there “should” be a better way :slight_smile:

Hey Pat -

Thanks for this, very helpful.  A few thoughts:

1) Your approach seems risky unless you are very careful about which pages you are modifying in each feature.  For example, what do you do when “Feature A” and “Feature B” are worked on simultaneously and both need to modify “Page ABC”?  Using page packs would make this very difficult to merge properly.

2) You mentioned each page pack has its own “module” name.  Did you mean Skuid “Module”?  If so, how do you reconcile these upon merge to CI/UAT/etc?  In other words, if Feature A gets a module name of “Feature A” then all pages will be in that Module.  What if this feature needs to modify a page that already exists in a different module?  Or what if this feature is creating a new master page or shared page of some type that another feature at some point needs?  Do the pages remain in Feature A Module? 

3) If you were to use the CLI (I understand you don’t want to so just humor me :)), do you think there would be other challenges you’d face if you just swapped in CLI to your process instead of page packs?

4) With your current process, what challenges are you facing specifically?

Never responded to this.

We’ve settled on working on Skuid pages one at a time.

I see challenges in using CLI since most of the developers i work with aren’t coding developers. They never use CLI. They are declarative developers like myself. The Gearset UX is way way easier to learn and execute on.

Page packs are required in order to get the pages into other orgs via source control. Do Skuid Grunt or Skuid CLI tools alleviate this? If so how?

Hey Pat - Yes, Skuid CLI (or Grunt) provide a mechanism to handle individual pages and pull/push to any org.  After “pulling” the pages from the org, you use your VCS of choice to commit them to source control.  See the readme at https://github.com/skuid/skuid for more details on how to use.

I guess what I’m missing is an understanding of being able to view the Skuid page XML as metadata so we can commit, review and merge changes to the page. Such as adding an action to a button. Right now, the only thing I see options for is to replace the page in it’s entirety.