Custom Metadata - A better way to manage Skuid page/pageassignment/etc. migrations?

Barry SchnellBarry Schnell πŸ’ŽπŸ’Ž
edited June 25, 2020 in Ideas
As discussed in the community in the past, managing migrations of Skuid pages/pageassignments/etc. can be difficult. Β Additionally, tracking pages in a VCS is challenging and often times, what I've heard is that a lot of users building Skuid based pages/apps have a dedicated sandbox org that always contains the "truth" (aka. latest version).

With the introduction of Custom Metadata in Summer '15 and enhancements coming in Winter '16 (native ui support, enhanced permission management, etc.), my thinking is that this might be a great solution to overcoming some of these challenges.

I'm curious if leveraging Custom Metadata would be a fit for pages, page assignments, etc., if this has been explored/evaluated yet and if so, what the future might hold?

Thanks!
1
1 votes

Not Planned Β· Last Updated

Β«1

Comments

  • edited March 1, 2017
    We would love to move to Custom Metadata for Pages, Page Assignments, Model Services, Model Service Adapters and other things. It's probably something we'll be exploring in the next Major Release after Banzai. The biggest issue right now is their lack of support for any long text. So we won't be able to move Pages until something like that is supported.
  • edited March 2, 2017
    There are 3 major reasons why we cannot use Custom Metadata types yet for Skuid, much as we would like to:
    1. No Long Textarea (LOB) fields --- Skuid Page XML metadata is stored in a combination of several Long Textarea fields on the Page object. Because Custom Metadata types do not offer Long Textarea fields, we really can't use them right now as a viable alternative for storing Skuid Page metadata.
    2. No Relationship / Lookup fields --- the Skuid Page Version and Page Assignment objects have lookups / master-detail relationship fields to Page records. And with Banzai, the Page object has a Lookup field to Page, to allow for Master Page functionality. Custom Metadata Types do not have a concept of Relationship fields.Β 
    3. No Triggers on Custom Metadata types --- the Page and Page Version objects have trigger logic that is essential for their functionality. Custom Metadata types do not support triggers.
    4. No DML support on Custom Metadata Types --- currently we use DML to create/modify/delete Skuid objects / custom settings. Custom Metadata Types can only be edited with the Metadata API. This presents substantial migration challenges for our existing codebase, including creation / updates of records from our Post-Install Apex script, modification of pages from the Skuid UI / Composer... it would take a lot of complex migration work.
    In short, we are functionally unable to migrate any of the core Skuid objects to Custom Metadata types in the foreseeable future. Some of the new configuration objects we have added with Banzai could be migrated to Custom Metadata types, but we do not have immediate plans to do this.

    I have heard that Relationship fields are up soon on the Custom Metadata Types roadmap, and after that LOB fields, but no timelines have been disclosed yet.
  • Barry SchnellBarry Schnell πŸ’ŽπŸ’Ž
    edited February 25, 2017
    Thanks for the insight Ben/Zach, makes sense why moving there now isn't possible but glad to hear that it's a potential future direction once MDT matures. Β Having this in place would make things soΒ much easier in terms of deployment management, version control, etc.

    I've read the roadmap includes Relationship/Lookup Fields and APEX DML support, hopefully LOB & Trigger support will be included as well. Β Though not ideal, I could see ways to overcome the lack of trigger support but lacking LOB would be make things difficult, or at the very least, extremely messy.
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited June 25, 2020
    So I've been setting up Source Control in BitBucket. One thing that would make this setup much better would be if Skuid were to use Custom Metadata.

    I've looked at the list of the 4 issues provided and it seems all but "No Triggers on Custom Metadata types" is now available. Is there any update on this? A workaround maybe?

    I know that static resources can be used to commit changes to source control, but the issue with this is that there is still a very manual process to ensure there are no overwrites. Can't very well pack everything into one static resource, so steps must be taken to pack changed pages into their own static resource.
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited June 25, 2020
    Any tips and tricks anyone would recommend on this?

    My current strategy is clone the Skuid Page into new Record/Named version and update Visualforce. Allows for one page pack everytime.
  • edited June 25, 2020
    Pat, any reason why you're not using Skuid Grunt or Skuid CLI (https://github.com/skuid/skuid) to do source control on Skuid Pages? This is our recommended approach. When doing active development on Skuid Pages, I would definitely use one of these two tools instead of Page Packs.
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited September 29, 2017
    Looking to avoid using any command line as it's far from user friendly when client declarative developers get involved.
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited September 29, 2017
    Additionally, it's not doing a commit to a branch so a pull request would not see the differences. Code cloberring will occur using only this CLI.
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 2, 2017
    Hmmm ... I suppose one could stage and commit the skuid pages to the branch and subsequently use Source Control this way. Not ideal as most declarative devs are not going to easily take this on, which is to say most Skuid devs are not going to.

    This doesn't fit the whole Skuid mantra of clicks not code (CLI) though.

    Any suggestions on how best to manage Skuid within Source Control with clicks not code?

    We're using Jira, BitBucket and Gearset to manage Agile Dev, Source Control and Release Management. We can click to create branches, commit changes, pull request, merge and deploy all from the cloud. Only wrinkle is Skuid Pages as they don't transfer via metadata without Static Resources and this would additional steps in order to commit to branch and deploy to Salesforce. ie. pack and unpack.

    If we had custom metadata, then everything would be a single step for each operation in the SDLC.

    Thoughts?
  • Matt SonesMatt Sones πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 2, 2017
    It's been an uphill battle, but we're doing everything on the command line now using bitbucket with skuid-grunt and grunt-ant-sfdc.

    A clicks-only approach would definitely make it easier for new hires to get working quickly.
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited June 25, 2020
    How about mirroring (auto saving) each Skuid page, page assignment and custom theme as a Static Resource? That way none of this malarkey about command line needs to be done.

    Seems too simple to me, but can this be done?

    More and more tools allow for simplified SDLC processes. We're using Gearset and BitBucket, so we can do everything from the cloud. ie. no local repository or command line.
  • Barry SchnellBarry Schnell πŸ’ŽπŸ’Ž
    edited June 25, 2020
    Hey Pat -

    I'm with Zach in that cli tools are the best technical solution available currently.Β  Hopefully Custom Metadata based Skuid is in the future :)

    That said, you asked for a way to avoid using cli.Β  So while it wouldn't be recommended, here's a couple of thoughts.Β  I haven't completely flushed these out in my head but I think they could be made to work:

    1) Build a GUI based tool on top of the CLI tools.Β  This avoids your users having to use the CLI but functionally still uses it.Β  You could write the tool in any language you choose (Java, c#, etc.) and make it portable for different operating systems.Β  This would accomplish your goal of "clicks" for the users that are maintaining pages.Β  They would just need to remember go "click" in the GUI after they are done making changes.

    2) Using a trigger/flow/process builder or some other SFDC feature, when a record changes, use an HTTP callout to commit that change to your VCS.Β  If I'm remembering correctly, bitbucket supports a REST API for various actions that can be taken.Β  You can leverage that such that when someone "saves a page", it invokes a back-end process to commit that change to your VCS.Β  You could get really fancy here and use something like Mulesoft or Jitterbit (or some other ESB type product) that have orchestration capabilities to do more than just "commit" something.

    3) Similar to #2, build a "sync" tool that takes data from Pages object and sync to your VCS.Β  You would build a queu'ing mechanism that would put a message in the queue indicating that someone saved a page, which org the save was in, what the record id of the save was, etc.Β  Then, your sync tool would poll for messages, grab the correct version of the page and sync to your VCS.

    4) You could write a browser extension that the user could interact with.Β  The extension would "save" the skuid page and make callouts to the VCS, etc.

    Again, just thinking out loud but hopefully this gives you some direction to potentially find a solution to avoid using CLI.Β  One thing to keep in mind with all the above is that some of them are better than others with regards to making the "save & commit to VCS" atomic - meaning that you are guaranteed to get your commit whenever someone saves.

    Tthought of 1 more...

    5) You could submit a feature request to Skuid to add extensibility to the Page Builder so that you can put your own "Save" button on it which would save the page and do the callout :)
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 3, 2017
    Thoughts on creating a Skuid page trigger to save the page to a static resource? This way it can be managed the same way as all other metadata in our SDLC processes. Of which no local repository or CLI is needed.
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 3, 2017
    And what's the downside of not using CLI?
  • Matt SonesMatt Sones πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 3, 2017
    I suppose that if all you're doing with VCS is record keeping, then the trigger for creating a static resource would be fine. But you're not going to be able to make changes to the resource and send it back to skuid that way (you could send them back to the resource in salesforce, but it wouldn't impact the skuid page at all).
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 3, 2017
    It would be easy enough to add code for the Skuid page builder to check for an update on open. I'd simply like a simplified approach for the strictly declarative devs. Which, again, is most Skuid devs.
  • Matt SonesMatt Sones πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 3, 2017
    Sounds exciting... but you want to actually change the page without opening the builder, right? It's almost like you're asking skuid to use your generated static resource as the page xml instead of the Page Layout text fields?

    Also, Gearset look awesome!
  • edited October 3, 2017
    I Am Jesus and I want a raise
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 3, 2017
    Hmmmm .... that would pretty slick, but I'm not sure it would be enough as there are other settings stored in other fields for the page.Β 

    Gearset is awesome for sure. Great fit for Skuid devs as it enables a declarative solution for source control solelyΒ  based in the cloud. No CLI or local repos. Compare and deploy between Salesforce org, Local Repo and Source Control.

    image



  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 12, 2017
    Zach and/ or Barry,

    I'm deep into an implementation of SDLC/Agile Development. The solution we're using is to use page packs as it's easily accessible to understood by all the developers. Developers that don't currently use any CLI tools I might add. Adding a process that makes use of CLI makes the implementation that much more intimidating to myself and our developers.

    What I'm trying to drive at is that CLI is an imaginary line in the sand that many declarative devs aren't easily/willingly going to cross. It's a piece of cake for the likes of anyone with a software engineering background/career.

    Skuid devs are commonly declarative devs who absolutely love what Skuid allows them to do sans any coding. Me clearly among them for sure.

    So coming back to my implementation. Page packs is a solution that works with source control. Not ideal but it works. Any suggestions on how best to use page packs in this SDLC flow starting with the Admin (Skuid dev)?

    image

    Thanks,

    Pat


  • Matt SonesMatt Sones πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 12, 2017
    I'm very interested in the outcome of this discussion! We've been fiddling with our deployment process, too.
  • Barry SchnellBarry Schnell πŸ’ŽπŸ’Ž
    edited October 12, 2017
    Pat -

    Happy to try to provide some thoughts on this but first, a few questions:

    1) What are you goals/objectives of the flow you describe?Β  For example, are you simply trying to identify a path for deployment, are you wanting to be able to properly version, track and maintain your skuid pages (aka. source code), etc.Β 

    2) What other items (beyond SFDC metadata) are you maintaining in version control (e.g. javascript, html, readmes, static resource bundles, etc.)

    3) How many developers on the team?Β Β 

    4) How many simultaneous "feature branches" do you typically have?

    5) Who is responsible for merging feature branches to master and are you using pull requests?
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 12, 2017
    1. Yes to tracking Skuid pages in source code. BitBucket in our case.
    2. Here's the list of what we are maintaining in source control.
    image
    3. Currently have 5 developers on the team.
    4. Approx. 5 per sprint.
    5. I am responsible for the merging of branches to master and we are using pull requests.
  • Barry SchnellBarry Schnell πŸ’ŽπŸ’Ž
    edited October 12, 2017
    Thanks Pat, this is helpful.Β  As with a solution for anything (e.g. C# vs Java, SFDC of Dynamics or Sugar, etc.), there often are multiple options so the more informed on the use case, the better a recommendation/path forward can be.

    Given your answers above, I think Page Packs (PP) are going to going to create challenges for you, especially since one of your goals is source control management.Β  Page Packs are big long strings of "stuff" and therefore, from a VCS perspective, extremely difficult to "merge."Β 

    For example, let's say that in Feature Branch 1 (FB-1) you make a change to Page A (P-A) and in feature branch 2 (FB-2) you make a change to Page B (P-B).Β  FB-1 and FB-2 then have a page pack of all the pages in that org.Β  If you merge the entire file in to master branch (MB), whoever gets merged last is going to win since unless you are extremely diligent/careful in isolating just the minor changes in the big block of text in that file, you'll have to take the entire pack file itself on the merge which would result in losing changes from the previous merge.

    There are some solutions to overcome this but they all would require manual effort.Β  For example, you could have a dedicated sandbox (DS) that is the only place you create the page packs from.Β  When FB-1 work is complete, the developer would login to the DS and manually update their page.Β  Same for developer when FB-2 is complete.Β  Then, you create the page pack from DS.Β  Alternatively, developers can communicate on a regular basis and say "Hey, Developers, I just updated my page, please update your org with the copy from my org for Page 123"

    The goal of any SLDC should be to eliminate as much manual effort as possible.Β  Given that and the challenges from above with regards to trying to manage PP directly from each FB, I think you're more than likely to end up in a scenario where you lose, break or forget a page.

    In short, given your desire to use source control and version pages, I wouldn't recommend using Page Packs.Β  If I was absolutely forced to using them, I would likely have a DS that would be the only "source" for generating the pack and each developer would be responsible for manually updating their page in the DS when they are complete with their work.

    I know it's now what you want to hear and I know CLI is a barrier of entry for your team, but if your goal is reliable version control, I do not see many other options short of the ones I mentioned in my previous post.

    Let me know your thoughts.
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 13, 2017
    My plan is to use named page packs for ds. The developer will update the module picklist with the fb name and that's what will be used to track the pages in source control. Thoughts?
  • Barry SchnellBarry Schnell πŸ’ŽπŸ’Ž
    edited October 13, 2017
    Not sure I follow you completely, sorry.Β  Can you explain step-by-step what you're thinking?
  • Bill McCulloughBill McCullough πŸ’ŽπŸ’Ž
    edited October 13, 2017
    Pat,

    You might want to look at Blue Canvas (https://bluecanvas.io/) as another option for simple source control of Salesforce.

    If you want to trigger an external system when a page changes, you can also look at Salesforce's Platform Events.Β  They just released this in Summer 17.Β  Β It will create an immutable object for each change of any object.Β  It publishes this change on a 'bus' that you can have an external application 'subscribe to'.Β  This link has a nice overview:Β Β https://developer.salesforce.com/blogs/developer-relations/2017/05/first-impressions-platform-events...

    As far as where you want to go with your setup to manage Skuid page changes in source control, I think you should go down the route of creating a Static Resource per Skuid Page/Assignment/Theme.Β  This will make it easy to merge individual items if needed and you can granularly move items from Dev to Full to Prod.Β  This means you'll need a trigger to create/save the static resource and an invocable action that you can run from a Skuid button to pull all changes from the static resource just moved into a new org.Β  Note that in my 'design' you'll need to watch the Static Resource for changes and have them committed to Source Control.Β  This is something that BlueCanvas does out of the box.

    Thanks,

    Bill
  • Pat VachonPat Vachon πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 13, 2017
    We create a sandbox, branch and when need be a static resource for the Skuid page(s) for each feature using Jira's issue key to name them all. The process works similarly to Bills, but it doesn't have the automation.Β 

    Hey Bill. Can I see your setup works? What do you use to commit changes from the sandbox to the feature branch?
  • Matt SonesMatt Sones πŸ’ŽπŸ’ŽπŸ’Ž
    edited October 13, 2017
    Blue Canvas looks nice. Not as far along as Gearset, but the continuous monitoring and deployment functionality could make life much easier.

    Anyone care to develop the triggers and invocable methods that Bill suggested? Or at least get us started?
  • Barry SchnellBarry Schnell πŸ’ŽπŸ’Ž
    edited October 13, 2017
    Pat - So at which point do you create the page pack?Β  How many Skuid pages does your org typically have? I'm still not following the entire process that you're using, sorry.Β  Can you walk it through step-by-step?Β  For example:

    1) Feature is requested
    2) Issue gets created in JIRA
    3) SCRUM occurs and feature gets assigned
    4) ....

    Bill - BlueCanvas looks intriguing, thanks for pointing this out.Β  It appears that it automates some of the processes that we've been discussing here with regards to keeping org & VCS in sync - it's the build vs. buy decision and I'm always a big fan of "buy" vs "build" :)Β  On the downside, the challenge with having a static resource for every page/assignment/theme is that you could quickly end up with lots of these.Β  In the last Skuid app I worked on, we had hundreds of pages.Β  Do you have any experience with using your 'design' in larger orgs with lots of Skuid pages?

    Matt - (Note: Friday humor coming in 3...2...1) Probably not what you were thinking when you asked to 'get us started' butΒ  here's a start :)
    trigger trgPageToVCS on skuid__Page__c (after insert, after update, after delete, after undelete) {
    Β  Β // create static resource for page
    Β  //Β  or publish event/make outbound call to 'notify' background process to grab latest page
    }
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!