MuleSoft provides many connectors to help businesses develop API-led connectivity solutions. Of all the connectors, the “For Each” loop is commonly used. This connector iterates through each element of a collection – in other words, each element of an array.
There are two different For Each connectors provided by MuleSoft: the “For Each” and the “Parallel For Each.” These have different functionalities and serve different purposes. Understanding when to use each one is crucial for the success of the APIs you are building.
When to Use MuleSoft For Each
The MuleSoft For Each connector gets an array of elements and processes – one at a time. So if there are 60 elements and each one takes a second to process, the total run for this connector is one minute. That’s something to keep in mind when using this connector as it could slow down your entire flow.
This is why the For Each connector should only be used with a small number of elements, or when performance isn’t critical.
When using this For Each connector, keep in mind that if one element fails to process, the connector immediately stops processing the rest of the records and moves on to the next connector. For example, let’s say that out of 60 records the connector fails at record 29. The remaining 31 records will not process as the connector has already failed.
Another important point to note – this connector doesn’t return a processed payload. Whichever payload is given to this connector gets passed through to the next connector.
In the above example, the Collection field in For Each is set to iterate over an array stored in payload.topics.
Key MuleSoft For Each Takeaways
The MuleSoft For Each connector should be used when:
- There are a few items to loop for per call.
- Performance isn’t crucial.
- One payload fails and the rest should stop processing.
- A processed payload isn’t returned.
When to Use MuleSoft Parallel For Each
Since the For Each connector doesn’t return a processed payload, one of the common patterns with the connector is to use Object Store to store the processed data for each element. This can be used successfully, but is not recommended since the Object Store has limitations on how many records it can process on a timeframe. It also can fail if you try to get elements at a pace faster than its limits.
One of the ways to avoid this pattern is to use the Parallel For Each connector. With this connector, each element is processed individually – the end result sees all of the payloads for each processed element being merged into one payload. You can then iterate through each one and run the logic that’s needed to run on each element. This case can’t be used for all scenarios, but it’s a way to avoid the Object Store pattern.
Parallel For Each works by processing each element in its own thread. There is a limit on how many threads it can run at one time (see the links below for more info). You can also set the limit yourself, which gives you flexibility and control over the number of threads it can run at one time.
Parallel For Each processes all records and the result is their payload – regardless of failure or success – returning the same number of payloads to match the elements given. This is useful when you want all records to process regardless of the status. You can then use your own logic afterwards in terms of what to do with success and failure payloads.
Another key component to the Parallel For Each connector is that the end results are returned as a combined payload – it actually returns the payload and you can do with it whatever you want. The For Each connector doesn’t return a payload and you have to use a variable or Object Store to store and retrieve the payload processed on the connector.
Key Parallel For Each Takeaways
The MuleSoft Parallel For Each connector should be used when:
- You avoid using Object Store in a loop, since Object Store drops messages sometimes.
- Performance is necessary.
- All should be processed, regardless of whether one of them fails.
- You want to see the end result of all elements passed.
For Each vs. Parallel For Each
| Use Case Requirement | For Each | Parallel For Each |
|---|---|---|
| Few items to loop for per call. | ✔️ | |
| Performance is critical. | ✔️ | |
| If one payload fails, the rest should stop processing. | ✔️ | |
| All payloads should be processed, even if one payload fails. | ✔️ | |
| When a processed payload doesn’t need to be returned. | ✔️ | |
| Visibility of the end result of all elements returned. | ✔️ |
Summary
Both For Each loop connectors are great tools for building a MuleSoft API. Each project will have different requirements. Based on your own project requirements, you will now be able to figure out which connector to use.
Resources:
- The For Each Scope
- The Parallel For Each Scope
- Set Up Your Salesforce Org for Integration with MuleSoft