Custom Metadata - A better way to manage Skuid page/pageassignment/etc. migrations?

Hmmmm … that would pretty slick, but I’m not sure it would be enough as there are other settings stored in other fields for the page.

Gearset is awesome for sure. Great fit for Skuid devs as it enables a declarative solution for source control solely based in the cloud. No CLI or local repos. Compare and deploy between Salesforce org, Local Repo and Source Control.

Zach and/ or Barry,

I’m deep into an implementation of SDLC/Agile Development. The solution we’re using is to use page packs as it’s easily accessible to understood by all the developers. Developers that don’t currently use any CLI tools I might add. Adding a process that makes use of CLI makes the implementation that much more intimidating to myself and our developers.

What I’m trying to drive at is that CLI is an imaginary line in the sand that many declarative devs aren’t easily/willingly going to cross. It’s a piece of cake for the likes of anyone with a software engineering background/career.

Skuid devs are commonly declarative devs who absolutely love what Skuid allows them to do sans any coding. Me clearly among them for sure.

So coming back to my implementation. Page packs is a solution that works with source control. Not ideal but it works. Any suggestions on how best to use page packs in this SDLC flow starting with the Admin (Skuid dev)?

Thanks,

Pat

I’m very interested in the outcome of this discussion! We’ve been fiddling with our deployment process, too.

Pat -

Happy to try to provide some thoughts on this but first, a few questions:

1) What are you goals/objectives of the flow you describe?  For example, are you simply trying to identify a path for deployment, are you wanting to be able to properly version, track and maintain your skuid pages (aka. source code), etc. 

2) What other items (beyond SFDC metadata) are you maintaining in version control (e.g. javascript, html, readmes, static resource bundles, etc.)

3) How many developers on the team?  

4) How many simultaneous “feature branches” do you typically have?

5) Who is responsible for merging feature branches to master and are you using pull requests?

  1. Yes to tracking Skuid pages in source code. BitBucket in our case.
  2. Here’s the list of what we are maintaining in source control.
  3. Currently have 5 developers on the team.
  4. Approx. 5 per sprint.
  5. I am responsible for the merging of branches to master and we are using pull requests.

Thanks Pat, this is helpful.  As with a solution for anything (e.g. C# vs Java, SFDC of Dynamics or Sugar, etc.), there often are multiple options so the more informed on the use case, the better a recommendation/path forward can be.

Given your answers above, I think Page Packs (PP) are going to going to create challenges for you, especially since one of your goals is source control management.  Page Packs are big long strings of “stuff” and therefore, from a VCS perspective, extremely difficult to “merge.” 

For example, let’s say that in Feature Branch 1 (FB-1) you make a change to Page A (P-A) and in feature branch 2 (FB-2) you make a change to Page B (P-B).  FB-1 and FB-2 then have a page pack of all the pages in that org.  If you merge the entire file in to master branch (MB), whoever gets merged last is going to win since unless you are extremely diligent/careful in isolating just the minor changes in the big block of text in that file, you’ll have to take the entire pack file itself on the merge which would result in losing changes from the previous merge.

There are some solutions to overcome this but they all would require manual effort.  For example, you could have a dedicated sandbox (DS) that is the only place you create the page packs from.  When FB-1 work is complete, the developer would login to the DS and manually update their page.  Same for developer when FB-2 is complete.  Then, you create the page pack from DS.  Alternatively, developers can communicate on a regular basis and say “Hey, Developers, I just updated my page, please update your org with the copy from my org for Page 123”

The goal of any SLDC should be to eliminate as much manual effort as possible.  Given that and the challenges from above with regards to trying to manage PP directly from each FB, I think you’re more than likely to end up in a scenario where you lose, break or forget a page.

In short, given your desire to use source control and version pages, I wouldn’t recommend using Page Packs.  If I was absolutely forced to using them, I would likely have a DS that would be the only “source” for generating the pack and each developer would be responsible for manually updating their page in the DS when they are complete with their work.

I know it’s now what you want to hear and I know CLI is a barrier of entry for your team, but if your goal is reliable version control, I do not see many other options short of the ones I mentioned in my previous post.

Let me know your thoughts.

My plan is to use named page packs for ds. The developer will update the module picklist with the fb name and that’s what will be used to track the pages in source control. Thoughts?

Not sure I follow you completely, sorry.  Can you explain step-by-step what you’re thinking?

Pat,

You might want to look at Blue Canvas (https://bluecanvas.io/) as another option for simple source control of Salesforce.

If you want to trigger an external system when a page changes, you can also look at Salesforce’s Platform Events.  They just released this in Summer 17.   It will create an immutable object for each change of any object.  It publishes this change on a ‘bus’ that you can have an external application ‘subscribe to’.  This link has a nice overview:  https://developer.salesforce.com/blogs/developer-relations/2017/05/first-impressions-platform-events…

As far as where you want to go with your setup to manage Skuid page changes in source control, I think you should go down the route of creating a Static Resource per Skuid Page/Assignment/Theme.  This will make it easy to merge individual items if needed and you can granularly move items from Dev to Full to Prod.  This means you’ll need a trigger to create/save the static resource and an invocable action that you can run from a Skuid button to pull all changes from the static resource just moved into a new org.  Note that in my ‘design’ you’ll need to watch the Static Resource for changes and have them committed to Source Control.  This is something that BlueCanvas does out of the box.

Thanks,

Bill

We create a sandbox, branch and when need be a static resource for the Skuid page(s) for each feature using Jira’s issue key to name them all. The process works similarly to Bills, but it doesn’t have the automation. 

Hey Bill. Can I see your setup works? What do you use to commit changes from the sandbox to the feature branch?

Blue Canvas looks nice. Not as far along as Gearset, but the continuous monitoring and deployment functionality could make life much easier.

Anyone care to develop the triggers and invocable methods that Bill suggested? Or at least get us started?

Pat - So at which point do you create the page pack?  How many Skuid pages does your org typically have? I’m still not following the entire process that you’re using, sorry.  Can you walk it through step-by-step?  For example:

1) Feature is requested
2) Issue gets created in JIRA
3) SCRUM occurs and feature gets assigned
4) …

Bill - BlueCanvas looks intriguing, thanks for pointing this out.  It appears that it automates some of the processes that we’ve been discussing here with regards to keeping org & VCS in sync - it’s the build vs. buy decision and I’m always a big fan of “buy” vs “build” :)  On the downside, the challenge with having a static resource for every page/assignment/theme is that you could quickly end up with lots of these.  In the last Skuid app I worked on, we had hundreds of pages.  Do you have any experience with using your ‘design’ in larger orgs with lots of Skuid pages?

Matt - (Note: Friday humor coming in 3…2…1) Probably not what you were thinking when you asked to ‘get us started’ but  here’s a start :slight_smile:

trigger trgPageToVCS on skuid__Page__c (after insert, after update, after delete, after undelete) {<br />&nbsp; &nbsp;&#47;&#47; create static resource for page<br />&nbsp; &#47;&#47;&nbsp; or publish event/make outbound call to 'notify' background process to grab latest page<br />}

Brilliant, @Barry! That’s exactly what I needed! :wink:

On a serious note… what 's wrong with having lots of static resources?

Glad you enjoyed Matt :) 

Technically, there’s nothing wrong with lots of static resources per se.  However, the more “stuff” your managing via non-native processes the more risk you introduce in to your overall solution.  Additionally, org refreshes with dev tools, etc. become slower as do deployments (more files to sync) thereby decreasing developer efficiency.  Lastly, there are governors on static resources (no file larger than 5MB & 250MB total).  Technically it can work, it just introduces variables and in-turn risk that would otherwise be unnecessary.

I do understand and appreciate your guys desire to find declarative solutions to manage Skuid Pages with a VCS and there are some solutions that will work.  The main challenge is that there isn’t an available tool today that does exactly what you guys want/need and therefore a custom solution is required to facilitate it.  While this can be done, that solution needs to be maintained & supported which takes away from focusing on running your business and adding value to that business.  Personally, I just think that the increased “risk” and “overhead” associated with any of those processes isn’t worth the “reward.”  That said, happy to help continue to find possible solutions to the problem - i just wouldn’t recommend any of them :slight_smile:

The idea of just notifying the VCS (or some kind of system) to run ‘skuid-pull’ for the changed page seems ideal to me.  Are there servics that could do that kind of thing?

I’m completely out of my depth here. I’ve picked up skuid-grunt and use the CLI for skuid and git with bitbucket (turns out the commands are much faster than using SourceTree).

We’ve been pounding away at trying to automate some kind of deployment system that would work well for both salesforce metadata and skuid data.

Let me try to come up with a fancy picture to demonstrate what our system looks like, and then maybe you folks can offer some suggestions?

That would be very helpful Matt.  The more details on what you guys currently have process & tools wise the more meaningful our discussions around them can be.

Regarding just notifying the VCS to run the CLI commands, if it were me and I was absolutely forced to find a declarative solution, I would target the absolute minimum to get the job done.  This would mean, as you suggest, finding a simple way to initiate a “skuid-pull/push/etc.” when a page is changed.  In my other post, I mentioned creating a GUI wrapper on top of the CLI tools or a browser extension - either of these would probably be where I’d start.  This avoids having to do anything on the SFDC side (e.g. triggers, events, etc.) and instead, just requires the developer to remember to push the button in the GUI when they are done developing their page.  The GUI could use SFDC apis and SOQL to query for information, pages, etc. and present the user with a drop down that allows them to “pick” which page they want to “commit to VCS”. The browser extension could just take the current page information and initiate the CLI commands although this gets a little complicated due to security limitations from within a browser.

Ok, here’s approximately what we’re doing at the moment…
It’s a manually intensive process, which is mostly working, but we run into hiccups primarily with merging SFDC metadata between orgs after a git merge.

We were attempting to use grunt to create an automated deployment process–basically looping through all our customers and running all the commands that we run on the command line in the right sequence–but it’s fraught with opportunities for failure.

Suggestions?

We don’t necessarily need a clicks-only solution (although it would make it easier for the over Dev on my team). I’m mostly interested in finding a way to better streamline or automate the deployment process.

Barry,

Nice start…where’s the Github/Gitlab/Bitbucket repo???  :slight_smile:

Unfortunately, I don’t have experience using my proposed design.  I was ‘threading’ off a Pat’s design and adding some options to the mix.  I have used Skuid Grunt, but primarily for page backup and check in to Git.  I have had the same thoughts as Pat in that there ‘ought to be a better way’.  I just haven’t pursued them in earnest.

I think BlueCanvas has the nicest out of box experience for declarative and Apex developers.  It keeps track of everything with limited fuss.

Thanks,

Bill