Posted on October 18, 2021 in Developers, Admins  

Salesforce Testing – Everything You Need to Know

Testing and test automation are essential parts of managing risk and delivering quality Salesforce releases on time. Getting it right requires a nuanced approach, and context is key in picking the right path. Because Salesforce is a very dynamic platform with change driven both by Salesforce’s expansion of the platform and by your customizations to meet business requirements, you need to get comfortable with the idea that test automation (and quality) is an ongoing journey.

This blog is a brief primer for Salesforce testing. We explore several key questions, provide an overview of the testing solutions available, and some simple tips to help outline the basics. Read on!

What is Testing in Salesforce?

Salesforce testing is what you would expect for any enterprise application, plus dealing with 3 major releases per year, plus the occasional patch from Salesforce.

• Job one is testing to make sure that your new custom features function as required and that nothing in your Salesforce Org, including integrations with other applications, gets broken with a new internal release.
• Job two is fitting Job one into a tri-annual major release cadence from Salesforce – making sure that nothing in a Salesforce update or patch causes any problems with your existing or new functionality.

This blog is focused on Salesforce testing related to making sure the application works as designed and as expected – so generally, unit testing, functional testing, integration testing, user acceptance testing and regression testing. Each of these areas is worthy of volumes. What’s not covered are security and performance testing. Both very important, but would put us way outside of our ‘coffee break’ target.

Let’s break the conversation into two pieces: 1) testing Salesforce major releases and 2) testing customizations and applications built on Salesforce.

Testing Salesforce Major Releases

Salesforce is a huge and innovative company. The challenge for most Salesforce customers is keeping up with releases and keeping track of the areas that affect the business. Salesforce does more than most to prepare customers for updates, but it’s still a massive undertaking. Depending on the size of your team and the amount of customization in your org, you’re going to need a little or a lot of help.

Salesforce does a great job of testing releases, so there’s no need (and it would require too many resources) to test everything. This is about keeping regression tests up to date and some targeted extra testing. It’s all about knowing where to look and picking your battles. Some guidelines:

• Target the features that are important to your business.
• Identify areas where you’re affected by automatic updates (Multi-Factor Authentication as an example).
• Plan on an ever expanding regression test – test what you built before, test what you just built.
• Test integrations with the applications in your Salesforce ecosystem.
• Do targeted smoke testing and/or exploratory testing where you don’t have coverage and experience says test anyway.

Tip: Find someone/some organization to help identify the key elements of major updates. The easiest way to start is to follow bloggers and organizations like SalesforceBen.com who regularly post highlights of Salesforce major updates. If you’ve got relationships with ISV’s and SI’s who have a built-in requirement to help their customers manage Salesforce updates, that’s another option. The main thing is that these folks will do a lot of the heavy lifting of reviewing and organizing the information.

Testing New Customizations and New Applications

When it comes to testing Salesforce customizations (including custom applications built on Salesforce), the picture gets more complicated. That’s because the responsibility for the quality of the customizations is not shared with Salesforce. There’s no vendor army working to make sure that your changes are going to succeed. Here are the key differences from Salesforce major release testing.

Not just QA – When testing your code, it’s not just QA. The teams involved have expanded to include architects, developers and business analysts, often each with their own testing tools and methodologies. The end goal is to get everybody on the same page and have test automation throughout the development lifecycle, but this is not average and is far from a foregone conclusion. Context is very important – staff skill sets, tooling, relative stages of devops are all considerations.

Full lifecycle testing – It’s an article of faith that bugs found earlier in the lifecycle cost less to fix. This is especially true for Salesforce applications because of the damage Salesforce related outages can cause. Testing throughout the lifecycle means you’ll need to include many more kinds of testing than what’s typically considered for major release testing. We’ll get into testing solution options for Salesforce below.

Continuous testing – It’s important to remember that, unlike regression testing for a Salesforce release, testing new development is not a linear process. The long term goal should be continuous testing – especially if you’re adopting an agile development methodology. Continuous testing and shrinking the test feedback loop as part of the full SDLC is the best way to avoid growing testing debt, increasing risk and the likelihood of a spectacular failure of your Salesforce application(s).

How Test Automation Can Help

Test automation won’t guarantee good outcomes, however, it is an essential part of any overall quality test plan. If you keep in mind some obvious but occasionally overlooked truths, you can avoid early pitfalls.

• Truth 1: A test must be repeatable and have a known outcome to be automated.
• Truth 2: Automate the areas where manual test execution falters – highly repetitive, highly complex, too many tests in too little time.

Where test automation helps most:

• Processes and flows that are used frequently
• Anything that is complex and/or has many permutations
• Whenever testing significantly benefits from rapid execution (API vs UI testing)
• Wherever mvp or mvq requires a large scope and/or volume of tests

Read more: What is Test Automation in Salesforce and Why Does it Matter?

Test Automation Benefits

If you’re able to make the shift from no testing or manual test execution to test automation, the benefits can be huge. Some are obvious – you’ll remove testing as a release blocker and release quality and business outcomes will improve. Some are more subtle but equally important. Here are a few highlights:

• Decrease the testing debt and lower risk.
• Improve the speed, coverage and accuracy of regression test execution (by several multiples vs. manual regression).
• Enable repurposing of resources to improve the coverage, adding tests for new features and optimizing the testing process (execution time, coverage, accuracy – e.g. risk/coverage/test/complexity).
• Add back exploratory testing. When we automate, we can create the time to explore those unknowns in the product/service/CX, to get a better understanding of the whole product. This is where testing as a creative art happens.
• Work life improvement for the whole team.Good automation improves confidence and quality. Bad automation is just ambiguity and tech debt.

Different Testing Solutions: What are my options?

Native Salesforce Testing

Testing options native to Salesforce are focused on the developer, mostly on unit testing. Salesforce includes tools in the Developer Console to create and manage tests, test classes, methods for creating and storing data, code coverage and more. There are trails and trail mixes with excellent guidance for setting up unit testing for LWC and Apex code. Here’s a trailmix that covers the basics. Developer-focused unit testing is a great start, but can be a difficult transition for businesses customizing their orgs primarily through declarative tools. Unit tests alone are not going to meet requirements for later in the lifecycle – especially UAT, integration and regression testing.

Manual Test Execution

Most organizations begin with manual test execution because it’s simple to set up, lots of people can be involved with very little training and it’s easy to share results. The problem with manual test execution is that it doesn’t scale, can be repetitive (aka boring), error prone and typically leads to testing debt and unmanaged risk over time.

Exploratory Testing

It’s important to make a distinction between manual test execution and exploratory testing. Although exploratory testing is a manual process, it’s more creative and technical than scripted, and it’s a very important part of filling in the gaps by testing outside of typical use cases. Test automation can’t (and shouldn’t) cover everything. There’s always something you’ll miss or decide not to automate along the way. When done properly, exploratory testing should inform future test automations.

Open Source Tools

Selenium is the go-to open-source framework for web application testing. It’s widely used and has a well-organized community. However, there’s a relatively high skill level required for proficiency and the tests, especially in a Salesforce context, require more than average maintenance because of the rate of change of the Salesforce platform (both updates and customizations).

Commercial Testing Tools

There is a wide variety of commercial tools available. A useful way to segment them is: 1) generic web app testing, 2) generic web app testing with Salesforce features, 3) testing tools custom built for Salesforce. The right choice depends on the organization. Is Salesforce the only platform to be tested, is there a QA team, is the QA team full of coders or manual testers, is there a well-established devops culture for Salesforce releases or is it more ‘push to production’ and ‘cross your fingers’? Context is everything. Key things to remember: There are no silver bullets. It’s ok to have multiple testing solutions. Plan ahead. Make sure the tool you choose has great vendor support and a vocal, connected community. At Provar, we’ve been a custom engineered test automation solution for Salesforce since 2014. Contact us for a demo today!

Non-Technical Considerations

It’s not just about picking the right tools. There are some key people-related factors involved in delivering quality releases. Great testing requires curiosity, critical thinking and communication. Curiosity to look past the test requirements and functional spec and understand the business rationale and user perspective. Critical thinking and experience to identify key areas of risk or where a design might fall short. Communication to make sure that developers, business analysts, the testing team and other stakeholders are all on the same page. All very important, but a blog topic for another day.

Top Tips for Test Automation

When organizations embark on a test automation journey, there is a natural inclination to ‘automate everything, go for 100% coverage, find and fix more bugs’. It sounds good and appears to promise quality software and great outcomes – or at the very least, move the organization in the right direction. The problem with this thinking is that a) you’ll never get there, b) all bugs are not created equal (fixing 10 critical bugs is more valuable to the business than fixing 50 non-critical bugs), c) the incremental cost of testing as you get closer to full automation or 100% coverage will far outweigh the business risk. This is true in spades with Salesforce. Planning and efficiency are huge success factors. The following tips should help you make the most of your testing resources.

• When developing a test automation plan, pay attention to context. All organizations are not the same. Testing skills, tooling, level of DevOps process and release quality history (successes vs. failures) will all play a part. Understand your organization’s level of testing maturity and plan accordingly.
• Automate what matters to your business. Gather requirements, identify areas of business risk, pick your targets.
• Develop, document and socialize the testing plan. Update and repeat, as needed.
• Build up your test automation over time (do not try to automate everything).
• Pay attention to test effort/business risk ratios. Don’t spend a lot of time developing and maintaining tests for areas of low business risk.
• Develop testing as a team sport. Share the testing culture across QA, developers and business analysts. Avoid testing siloes. Get the entire team involved at the beginning to foster whole-team ownership.

Summary

Testing and test automation are key to improving the quality of your Salesforce releases. Starting small, paying attention to the specifics of your business, test effort/risk ratios, existing tooling and people, are all part of the essential elements of success. Context is king.

There are a lot of resources available to help you get started or get to the next level in your Salesforce quality journey. The Salesforce community, vendors and system integrators are all ready to help you.

As an ex-manual tester at a company where new releases and features regularly broke our Salesforce applications, I’ve learned a lot of these lessons the hard way. Save yourself some time (and trouble) by looking at companies such as Provar, who specialize in Salesforce test automation. Or just use this primer as a guide to evaluate any of your choices when planning your own Salesforce test automation. Good luck!