Create a Best-of-Breed Enterprise Data Factory with Forceea

It’s impossible to imagine Salesforce without data. When we’re talking about generating testing (or “dummy”) data, you probably haven’t realized it’s a very complex and potentially expensive task.

In this article we’re going to work through a step-by-step guide – a workshop that will definitely bring huge improvements into the way you create data.

Use Cases (Everybody Needs Data)

A data factory is a set of tools that allow a user to generate data for some specific use cases:

• A developer needs to create data for Apex unit tests.
• A developer must populate scratch orgs with hundreds of records for easier development.
• An administrator or architect wants to populate a sandbox with thousands of records for UAT testing.
• An architect needs a sandbox with millions of records for testing SOQL queries of large data volumes.

Data from the Production org is (for many reasons) not a good idea, at least for scenarios A and B. Since a mature Salesforce project usually has complex data needs, an Enterprise Data Factory (EDF) is the data factory that enables organizations with complex data requirements to generate data easily and efficiently, allowing anyone (even with basic programming skills), to get the magic wand.

The Magic Wand (We Call It Forceea)

Forceea (forsi:a) is an open-source GitHub project I started in 2017. I hear you wondering what Forceea could do. The following list describes its most basic capabilities (there are many more for you to discover…)

• Inserts, updates and deletes records for standard or custom sObjects Big Objects, synchronously (for test methods) or asynchronously (for populating your org with millions of records).
• Supports all field data types of any standard or custom field.
• Supports the Dadela data generation language for field definitions.
• Can automatically define the required fields.
• Handles record types and field dependencies (dependent picklists).
• Supports record groups for inserting, updating and deleting records.
• Provides templates for constructing a DataFactory class that can be used for flexible data generation.
• Supports variables and functions.
• Validates the definitions based on the field data type.
• Has an extended error messaging system.

This article will be a step-by-step workshop, so log into your org (Developer, Scratch or Sandbox) and install the Forceea unlocked package with:

sf package install -w 10 -p 04t4I0000011V6UQAU -r -o <TargetOrg>

or visit the Forceea GitHub page and click the suitable button to install it.

Your Complex Scenario

Let’s suppose you have installed Forceea and you’re ready to create data for accounts, opportunities, and products. Of course, you have specific requirements for each object. For example, for accounts, you want to include the following fields:

• Industry: Random picklist values
• Phone: Random values like “(44) 7911 XXXXXX”
• Type: Random picklist values except “Prospect” and “Other”
• Website: Any random URL
• Site: The same URL as Website

 Additionally, you want to get:

• Products of the “Service” record type, with standard pricebook entries.
• Opportunities related to an account, closing between 30 and 120 days in the future.
• Opportunity products (opportunity line items) with the standard price book unit price and a random quantity between 1 and 10.

This complex scenario is a good crash test for any EDF for benchmarking its capabilities and flexibility. Get ready, our workshop is beginning now!

The DataFactory Class

First you must create your new DataFactory class (any name is fine, but you may agree that DataFactory is cool!)

public with sharing class DataFactory {
  // your Templates here
}

The best practice when you work with Forceea is to use templates. Here is the basic structure of any template:

public static Forceea.Template templateMyObjects() {
  // Template details
}

code

A Forceea.Template defines one or more sObjects, their field values, their number of records and many other details. A template can include other templates, which provides an encapsulation logic that is extremely flexible.

Technically a template is a Map<String,FObject> with:

• Key: A label you give for the FObject, usually in plural, e.g. “Accounts”
• Value: A FObject

Forceea uses the FObject class to keep all the details about an SObject. This is the standard “structure” of any template:

public static Forceea.Template templateMyObjects() {
  return new Forceea.Template()
    .add(MySObjects, new FObject(MyObject__c.SObjectType, 10)
 	// methods
  );
}

As you see, we “add” a new Template item with the key “MyObjects”. The value is a new FObject for MyObject__c, which should generate 10 records. So far, so good! But what are “methods”?

Well, with methods you can define your field definitions (and many other settings). Let’s take a closer look.

The Account Template

You start your data factory with the template for account:

public static Forceea.Template templateAccounts() {
  return new Forceea.Template()
    .add('Accounts', new FObject(Account.SObjectType, 10)
      .setDefinition(Account.Name, 'static value(Account)')
      .setDefinition(Account.Name,
        'serial type(number) from(1) step(1) scale(0)')
      .setDefinition(Account.Industry, 'random type(picklist)')
      .setDefinition(Account.Phone,
        'random type(phone) format("(44) 7911 dDDDDD")')
      .setDefinition(Account.Type,
        'random type(picklist) except(Prospect,Other)')
      .setDefinition(Account.Website, 'random type(url)')
      .setDefinition(Account.Site, 'copy field(Website)')
  );
}

The most important here is the setDefinition method. It takes 2 parameters:

• The sObject field, e.g. Account.Name,
• The Dadela field definition, e.g. random type(url)

Have a look into the Dadela field definitions in the above template. Can you guess what kind of data each definition returns? Let’s see them together:

• The static always returns the same value (boring, but essential).
• The serial type(number) generates numbers – here 1,2,3, etc. It can also create dates/datetimes, or values from a picklist/list, get the ID of a lookup sObject, or create unique combinations of multiple serial definitions (permutations).
• The random creates values that are randomly selected, like random picklist values (obviously from a picklist field), or random numbers, Strings, texts, date/datetimes, Booleans, etc. Random IDs (of an sObject related with a lookup or master-details relationship) are provided as well.
• The copy just copies the value of a field on the same record or another related record.

Enough with the theory! We’re going to proceed to the other templates now.

The Product Template

What are opportunities without products? We want products, so here is our template for them:

public static Forceea.Template templateProducts() {
  return new Forceea.Template()
    .add('Products', new FObject(Product2.SObjectType, 5)
      .setDefinition(Product2.IsActive, 'static value(true)')
      .setDefinition(Product2.Description, 
        'random type(text) minLength(30) maxLength(50)')
      .setDefinition('$StandardPrice',
        'random type(number) from(100) to(1000) scale(2)')
      .setStandardPricebookEntries('$StandardPrice')
    );
}

This template should be familiar to you. It has the same structure as the previous template for account, but here things are getting quite interesting! Observe the definition for the field $StandardPrice. You may ask yourself “Did the last Salesforce release introduce new Product2 fields?” Unfortunately, the answer is no!

This is a Virtual Field (it starts with a $): a field that doesn’t really exist, but it can get data to provide its services to another field. The Virtual Field here has a specific scope: to generate random values for the standard price book entries. Do you see the next method setStandardPricebookEntries? This method will get the values of the virtual field $StandardPrice and automatically insert entries in the standard price book. Cool, isn’t it?

Note: Go to the Price Books tab and make sure your standard pricebook is active!

But we haven’t finished with Product2 yet. You have two record types (Product and Service) and you want to insert products of the second one. How do you do this? With another template of course!

public static Forceea.Template templateServiceProducts() {
  return new Forceea.Template()
    .add('ServiceProducts', templateProducts().getFObject('Products')
      .setVariable('recordType', 'Service')
      .setDefinition(Product2.RecordTypeId, 
        'static value({@recordType})')
      .setDefinition(Product2.Name, 'static value(Service)')
      .setDefinition(Product2.Name, 
        'serial type(number) from(1) step(1) scale(0)')
      .setDefinition(Product2.ProductCode, 'static value(SRV-)')
      .setDefinition(Product2.ProductCode,
        'random type(string) minLength(5) maxLength(5) ' +
        'startWith(upper) include(upper,digit)')
    );
}

Templates are such a great tool because they are (very!) flexible.

• The templateProducts included just the basic field definitions that are common to all record types. Your new templateServiceProducts will include the missing fields. 
• Here you define the RecordTypeId (using its developer name).
• The last definition (for ProductCode) has 2 parts: a static and a random, which concatenate to provide the final data.

Another very important aspect of this template is the 3^rd line, starting with the add method. You would expect a new FObject, like in the previous templates, but you find templateProduct().getFObject(‘Products’). Why?

Having an encapsulation logic, the basic TemplateProducts will provide its FObject to any other template derived from it. But you need to modify this FObject to add the details of the service record type. That’s why we use the Forceea.Template method getFObject, to retrieve the FObject with the “Products” key from templateProducts and then add new field definitions, specific to our record type.

And a final comment regarding the above template: did you notice the setVariable method and the {@recordType} variable in the RecordTypeId definition? Forceea variables allow you to “isolate” parameters that shouldn’t be “hidden” in the Dadela scripts (e.g. a record type developer name), something that increases the script readability.

The Opportunity Template

Now, let’s go to the opportunities template:

public static Forceea.Template templateOpportunities() {
  return new Forceea.Template()
    .add('Opportunities', new FObject(Opportunity.SObjectType, 10)
      .setVariable('today', Date.today())
      .setDefinition(Opportunity.AccountId, 
        'serial lookup(Account) mode(cyclical) source(forceea)')
      .setDefinition(Opportunity.Name, 
        'copy field(AccountId) from(Account.Name)')
      .setDefinition(Opportunity.Name, 'static value(" - ")')
      .setDefinition(Opportunity.Name, 
        'random type(text) minLength(15) maxLength(20)')
      .setDefinition(Opportunity.StageName, 
        'random type(picklist) except(Closed Lost, Closed Won)')
      .setDefinition(Opportunity.Amount, 
        'random type(number) from(10000) to(100000) scale(2)')
      .setDefinition('$days', 
        'random type(number) from(30) to(120) scale(0)')
      .setDefinition(Opportunity.CloseDate, 
        'static value({@today})')
      .setDefinition(Opportunity.CloseDate, 'function-add field($days)')
    );
}

This template introduces some new capabilities of Forceea:

• The AccountId field gets the IDs of the accounts previously created by Forceea. If we have no more accounts to get, it goes to the 1^st one (mode=cyclical)
• The CloseDate has 2 definitions. The 1^st just sets the CloseDate equal to “today”. The 2^nd uses function-add to add the number of $days to “today”, practically moving today to a random number between 30 and 120 days in the future.

The OpportunityLineItem Template

And finally, the template for opportunity products:

public static Forceea.Template templateOpportunityLineItems() {
  return new Forceea.Template()
    .add('OpportunityLineItems',new FObject(
          OpportunityLineItem.SObjectType, 30)
      .setDefinition(OpportunityLineItem.PricebookEntryId,
        'random lookup(PricebookEntry) source(forceea)')
      .setDefinition(OpportunityLineItem.Product2Id,
        'copy field(PricebookEntryId) from(PricebookEntry.Product2Id)')
      .setDefinition(OpportunityLineItem.OpportunityId,
        'serial lookup(Opportunity) mode(cyclical) source(forceea)')
      .setDefinition(OpportunityLineItem.Quantity, 
        'random type(number) from(1) to(10) scale(0)')
      .setDefinition(OpportunityLineItem.UnitPrice,
        'copy field(PricebookEntryId) from(PricebookEntry.UnitPrice)')
    );
}

The Final Template

Now we have configured our templates, we need to create the final template templateOppsWithProducts that encapsulates all the other templates:

public static Forceea.Template templateOppsWithProducts() {
  return new Forceea.Template()
    .add(templateAccounts())
    .add(templateServiceProducts())
    .add(templateOpportunities())
    .add(templateOpportunityLineItems());
}

That’s it! This super-template is all we need to create data for all these objects. Let’s try it!

Inserting Records

You’ve gone a long way to construct your DataFactory class with all these templates. Now you’re going to use them to insert records (Forceea can update or delete records, but that’s out of scope in this article).

Having a template, your job is very easy:

FObject.setGlobalVerbose('debug');
DataFactory.templateOppsWithProducts()
  .insertRecords(true); // allOrNone=true

This statement uses templateOppsWithProducts to insert records synchronously. It’s ideal for inserting data in Apex unit tests. When you’re debugging your test methods, the best practice is to use FObject.setGlobalVerbose(‘debug’) before you insert the records. This static method “activates” the Forceea debug logs, revealing invaluable information about the data generation process.

• Execute the code and see the Debug Log.
• Forceea will validate your Dadela field definitions and if it finds any errors, will display error messages. Many details will be displayed, and you see that your records are created successfully.
• Repeat the execution in an anonymous window to get “real” records into your org.

Modifying a Template

And now the funny part! Let’s say that someone else has created templateOppsWithProducts for you. But you want to do some modifications, for example to add a new field in account and modify a field in opportunity. How difficult can it be?

Forceea.Template template = templateOppsWithProducts();

template.getFObject('Accounts')
  .setDefinition(Account.AnnualRevenue,
'random type(number) from(100000) to(1000000) scale(2)');

template.getFObject('Opportunities')
  .replaceDefinitions(Opportunity.StageName)
  .setDefinition(Opportunity.StageName, 'static value(Closed Won)');

template.insertRecords(true);

Actually, it’s a piece of cake!

• You get the “accounts” FObject from your template and you just add a field definition for the AnnualRevenue.
• The same for opportunity. Here you want to replace the contents of all field definitions of StageName with a new definition, so you use the replaceDefinitions method first and then the usual approach with setDefinition.
• Finally you insert the modified template.

It couldn’t be easier, don’t you agree?

Summary

The power of Forceea is based on the combination of the Dadela data generation language and its Templates. Together they will help you to construct a robust, dynamic and easily expanded Enterprise Data Factory.

There are a myriad of Forceea features that could not be included in this workshop article. All these details are included in the Forceea Success Guide.

I’m convinced that Forceea can be your assistant to every data needs you may have. I hope you enjoy it!