Salesforce GitLab Delta Deployment: The Easy Way

Being a Salesforce Technical Lead/Architect for the last five years, I have had my fair share of managing deployments. On a certain project, still using deployment sheets and change sets at the time, I would be working on a sprint deployment for around two days. Checking the components, their dependencies and validating the change set. Later on, that same project adopted Salesforce Developer Experience (SFDX) with scratch orgs, and it was working fine… for a while.

What we’re about to see in this article is a combination of different approaches that we have managed to put in place on my current project to have version control and automatic deployments from GitLab. Even though Salesforce focuses on using SFDX with scratch orgs, we’re going to discuss how to put version control in place for regular sandboxes (yes, no need to reload data every 30 days), as well as automating package validation and deployment. And even more, deploying only the Delta! Note: This approach would be suitable for a small to medium size project, 5-15 people, notably with a Release Manager.

Configuring Your Salesforce Environment

Let’s use our Salesforce UAT environment as an example here. This is our Partial Copy Sandbox on which the current sprint is tested. The goal here is to configure the connected app that will be used by GitLab in order to deploy to this Sandbox. In order to do so, the first step would be to set up a certificate and key to secure the GitLab/Salesforce connection.

Certificate and Key

Using OpenSSL, we generated a key using these three commands:

• openssl genrsa -des3 -passout pass:x -out test.pass.key 2048
• openssl rsa -passin pass:x -in test.pass.key -out test.key
• openssl req -new -key test.key -out test.csr

With the key generated, one more command is needed to create the certificate: openssl x509 -req -sha256 -days 700 -in test.csr -signkey test.key -out test.crt.

Keep this certificate in a folder close by, you’ll need it soon.

Connected App

Going back to your Sandbox, go to your Salesforce setup, and create a connected app.
In the connected app settings, check the Digital Signature option and upload the .crt file you generated. 

In the OAuth scope, make sure that the app has the following scopes enabled: 

• Access the identity URL service (id, profile, email, address, phone)
• Manage user data via APIs (api)
• Manage user data via Web browsers (web)
• Perform requests at any time (refresh_token, offline_access)
• Access unique user identifiers (OpenId)

The connected app should give access to the profile you want to use for your deployments. Additionally, make sure that the permitted users policy is set to ‘Admin Approved users are pre-authorized’.

This should be enough for your Salesforce Sandbox. Note that sometimes it does take a few minutes for this to work on Salesforce. A few minutes later verify that the digital certificate has been uploaded correctly – sometimes this does not happen.

Configure Your GitLab Project

On your GitLab Project, you need to configure the connected Environment (UAT in this case), the variables, and the pipeline.

Connected Environment and Variables

On the left panel of your GitLab project, go to the Operate tab, and click on Environments. Create a new one and set the External URL as the URL to your Sandbox; for example: 

Next, you need to set up your GitLab variables. Go to the settings tab, and click on CI/CD. Expand the Variables section, and create the following variables (be sure to uncheck protected/mask variable checkboxes): 

• SF_CONSUMER_KEY_UAT: This is Consumer Key from your Salesforce Connected App of the UAT Sandbox
• SF_USERNAME_UAT: The user you wish to use for your deployments.
• INSTANCE_URL_UAT: Your instance URL, ex: https://test–uat.sandbox.my.salesforce.com

With the environment & variables set up, next you need to figure out your branching model/strategy. 

Branching Model and Sandboxes

When you want to think of a branching model, the first few questions you have to answer are:

• How many people are working on this project? 
• What’s their methodology? 
• How frequently are they deploying to production?
• Do they have multiple releases in parallel?

A strategy that I have been following throughout my last two projects (the last project team size was around 50 people, the current around 15), was to allow parallel work using two release branches while basing my strategy on having one Partial Copy Sandbox, one Full Copy Sandbox, and one too many Developer Sandboxes.

The Partial copy would be used for grouping the Sprint Release tickets and testing them. Once the Sprint is validated, the Sprint Release would be deployed on the Full Copy Sandbox for end-to-end testing and integration testing.

Similarly, the Full Copy Sandbox – which is meant to be isoProd – could be used alone for the quick release or hotfix deployment.

Putting this into the Branching Model perspective, the UAT Branch would be linked to the Partial Copy Sandbox, and the Staging branch to the Full Copy. Remember that our aim is to deploy to the Sandbox anytime we push to its related branch. 

We would finally get the two following scenarios:

Sprint Release

Keep in mind that in this case, we are considering that the Master branch and UAT branch are identical at the start of the release. Plus, during the sprint, if you’re creating new branches belonging to the same release, you need to create it from UAT to avoid regressions.

Hotfix Release

Here we would be creating our Hotfix branch from Master (or Staging if it’s Iso Prod), doing our changes on our Dev Sandbox and then deploying to Staging, completing our testing, and finally deploying to production.

GitLab YML Configuration

At this point, you should have completed your Salesforce Connected App configuration, defined your deployment user, configured your GitLab Environment Variables, and confirmed your release/branching strategy.

Next, we need to configure the gitlab-ci.yml file. This is the file that manages the pipeline that would be launched to automate deployment (this is defined in the CI/CD tab in the GitLab project settings). We have defined our need to be the following: 

• Whenever we create a Merge Request between a Release Branch and the UAT branch, I want a pipeline that will validate my package with the UAT(Partial Copy) Sandbox.
• When I do Merge the Release Branch, I want the pipeline to deploy the delta.

In the Gitlab-ci.yml file, we used the SFDX image in order to connect to the Salesforce Sandboxes. Additionally, we used the sfdx-git-delta plug-in to generate the package.xml by comparing two branches – in this case we are comparing between the Release Branch and UAT. This package.xml file is used to define the components to deploy to the Salesforce Sandbox.

Validating Your Merge Request

1. The first step required for the pipeline is to authenticate to Salesforce using the connected app and the jwtkeyfile that we generated in the first section, we can do so by using the following command:
   – sfdx force:auth:jwt:grant –clientid $SF_CONSUMER_KEY_UAT –jwtkeyfile test.key –username $SF_USERNAME_UAT –instanceurl $INSTANCE_URL_UAT Note: The test.key file needs to be in the repository. 
2. Second step would be to create the package.xml file. To do so, we use the sgd plugin as follows:
   – sfdx sgd:source:delta –to “HEAD” –from “origin/UAT” –output
   In this case, we are comparing HEAD (the current branch/commit that is being pushed) to UAT to generate the differences in a package.xml file created in the repository.
3. Next, with the package.xml generated, the pipeline should validate the package with the Sandbox. We use the following command:
   – sf project deploy validate –manifest package/package.xml -o $SF_USERNAME_UAT
   We are using the latest version of the SFDX commands. This will validate if the package pushed is okay, and additionally it will run the test classes by default. For more details on SF commands, check out this resource.

Deploying

With the Merge Requested validated, the next step is to complete the merge to the UAT branch to launch the actual deployment pipeline. The trick here is that we can’t use the HEAD to UAT comparison, since at this point the UAT branch would already have the new changes. The solution is to use the HEAD to HEAD-1 comparison. This is why, before you perform the merge, it is essential that you squash the commits! By doing so, the HEAD-1 will contain all the modifications from all your commits. If you don’t squash them, you will only deploy the modifications from your latest commit.

In the Gitlab-ci.yml the commands would look like:

• sfdx sgd:source:delta –to “HEAD” –from “HEAD~1” –output .
• sf project deploy start –manifest package/package.xml -o $SF_USERNAME_UAT

Once this is run, it would be performing a quick deployment since your test classes have already been validated. And that’s it, you’re all done! 

Final Touches

To add Destructive changes, you can only use pre-destructive changes with the Deploy Start command, and not the Deploy Validate (for now). The SGD plugin would generate the destructiveChanges.xml file automatically, and you can directly use it, even if there is nothing to delete.

When you run the sf deploy validate command, it will automatically run all local tests by default. For my project this is fine because we only have around 100 test classes and that way we avoid test class regressions later on. Validation takes around 4 min, and quick deployment is around 1 min.

Lastly, this is a Gitlab-ci.yml example so you have an idea of what you’re looking for:

Gitlab code

image: salesforce/salesforcedx:latest-slim
stages:
  - test
  - deploy
before_script :
  - apt-get update
  - echo "y" | sfdx plugins:install sfdx-git-delta
validate_UAT:
  stage: test
  environment:
    name: UAT
    url: https://test--uat.sandbox.lightning.force.com/
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
  script:
    - sfdx force:auth:jwt:grant --clientid $SF_CONSUMER_KEY_UAT --jwtkeyfile test.key --username $SF_USERNAME_UAT --instanceurl $INSTANCE_URL_UAT
    - git fetch origin UAT
    - sfdx sgd:source:delta --to "HEAD" --from "origin/UAT" --output .
    - sf project deploy validate --manifest package/package.xml -o $SF_USERNAME_UAT 
deploy_UAT:
  stage: deploy
  environment:
    name: UAT
    url: https://test--uat.sandbox.lightning.force.com/
  rules:
    - if: $CI_COMMIT_BRANCH == "UAT"
  script:
    - sfdx force:auth:jwt:grant --clientid $SF_CONSUMER_KEY_UAT --jwtkeyfile test.key --username $SF_USERNAME_UAT --instanceurl $INSTANCE_URL_UAT
    - git fetch origin UAT
    - sfdx sgd:source:delta --to "HEAD" --from "HEAD~1" --output .
    - sf project deploy start --manifest package/package.xml -o $SF_USERNAME_UAT --pre-destructive-changes destructiveChanges/destructiveChanges.xml 

Gitlab code end

Summary

Overall, by combining what SFDX used with their scratch orgs, and the Git Delta plugin, we were able to create a branching model that uses standard Salesforce Sandboxes, while using an automatic delta deployment pipeline on GitLab. If you take a look at the entire Gitlab-ci.yml file, you’ll find that the entire file is around 25 lines, nothing too complicated. This would be most suitable for a medium size project with a Release Manager capable of managing the different releases branches and Sandboxes. Finally, a quick look at different options:

• Salesforce DevOps Center? Personally, given that I did try it out I felt it was not mature enough to be used. It has a great potential but it still needs some work, to adapt it to GitLab would be the first push forward.
• Change Sets? Having used them for a while on a big project, the difficulty to build them and deploy them is too much.
• SFDX with Scratch Orgs? I did have them on a previous project, not a bad approach but they require a team to keep them maintained. Creating the Scratch Org every month, and populating it. A bit too much work in my opinion.
• VSCode & SFDX without GitLab? Before using GitLab on this project, I was managing deployments directly from VSCode with the package.xml file. This could be an OK solution if you do not have GitLab. I had created a tool to generate the Package.xml file from a Confluence page or Excel deployment sheet in order to manage better.