The Information reported this week that OpenAI, Anthropic, and other AI labs are currently spending billions training large language models (LLMs) in reinforcement learning (RL) gyms, which are simulated versions of enterprise apps such as Salesforce.
The aim of this is to arm AI agents with the capability of performing real office tasks – The Information use prospecting in Salesforce, sending Calendy invites, or running financial models in Excel, as key examples of this.
While this may be a groundbreaking next step for AI, this does create potential threats to Salesforce’s primary AI offerings. If these LLMs learn to use Salesforce directly, they could potentially bypass what Agentforce can offer.
Of course, Salesforce has a reliable moat to keep their product strong. Proprietary data via Data Cloud, as well as the Einstein Trust Layer, keeps users engaged with their product. But AI labs training models to use Salesforce intuitively could be a risk if it makes Agentforce look like just a wrapper. Let’s take a closer look.
What Are RL Gyms?
RL gyms may sound like an abstract idea, but the aim is to simply create fake workplace apps and let AI models practice within them until they can complete a task as reliably as a human worker.
These simulated environments give these models the chance to click through a mock Salesforce instance and trial actions such as filtering leads or updating statuses.
The aforementioned companies are hot on the idea and are investing significantly in these training environments. Anthropic leaders have reportedly discussed allocating around $1B over the next year, while OpenAI expect these data-related costs (which include both RL gyms and human experts) to rise to $8B by 2030, per The Information.
Both companies are also hiring domain experts – from data scientists at NASA to medical residents – to show models how actual professionals complete these tasks.
This takes the goal for AI beyond just answering questions – it will become a proactive digital colleague that can work on the same apps that millions use every day.
Why Could This Threaten Salesforce?
For Salesforce, the rise of RL gyms introduces a very real challenge for the CRM giant. If AI models can actively log in, navigate Salesforce, and complete workflows at human level, it’s easy to predict some customers pushing back on paying for Agentforce (especially as some already are).
After all, why buy Salesforce’s AI layer if a third-party model can handle the same prospecting and follow-up tasks directly?
This is what many in the industry call a “wrapper risk”. Since the beginning of the AI era, we’ve seen entire categories of startups – especially those building AI-powered productivity tools – lose relevance instantly because the likes of OpenAI release native features that accomplish the exact same thing.
The concern is that Salesforce’s Agentforce could face a similar fate if enterprises view it as a thin wrapper on top of technology that’s coming from Anthropic, OpenAI, etc.
At the same time, it’s far from a perfect comparison. Salesforce customers still expect their org to be secure, compliant and reliable – three things that you’d expect Agentforce to have a handle on. Still, the fact remains: if AI labs continue training models to use Salesforce directly, then the pressure on Salesforce to differentiate their product will only grow stronger.
“The Race Is On”
Salesforce Ben Technical Content Director, Peter Chittum, warns that while OpenAI and Anthropic may try to use RL gyms to train agents on Salesforce, the reality is far more complex, given how customized most orgs are.
“With a highly customized org – which is precisely why customers make the plunge with Salesforce – you could imagine Anthropic and OpenAI could teach their agents to learn about the customizations in a customer’s org, just like some Salesforce ISVs are doing already,” Peter explained.
“But that won’t just magically work without there being enough documentation in the org itself for the agent to go on, or well-logged usage data, which is not something Salesforce provides to the customer natively.
“Without that, it’s unlikely that the training that they’re doing will work with any kind of consistency. Where this approach could work would be if it were to partner with some ISV or build an in-house Salesforce expertise to help those Salesforce customers build that semantic knowledge base for the agent. But again, we’re talking about something more than just the RL gym training.
“Bottom line, the big frontier model providers are coming for Salesforce. Whether they can breach Salesforce’s mote and the intractable state of so many Salesforce customers’ metadata is another question. But this is the same challenge Salesforce faces. The race is on.”
Final Thoughts
Agentforce has faced its fair share of skeptics, and pressure continues to grow on Salesforce. Investors are cautious, user adoption is slow, and now they have a potential threat to the integrity of their product.
RL gyms are, of course, in their early stages, which gives Salesforce plenty of time to consider how to differentiate Agentforce as not another wrapper, but a necessary AI product for all Salesforce customers.
