Salesforce is today introducing Agent Evals, API Evals, and advanced cost controls for its AI agent control plane, Agent Fabric, with hard spending limits that can cap usage within a budget.
The CRM company says that as models, APIs, costs, and systems change, issues can silently surface, but catching them requires continuous building, maintaining, and evaluating. Once problems are discovered, resolving them means tracking down the owner and getting approval to restrict the agent – as it continues producing harmful results while spending tokens. Let’s take a look at their new approach to preventing this.
New Agent Tools Explained
Salesforce is introducing Agent Evals, which automatically generates tests before deployment and then evaluates agents, LLMs, and MCP servers in production. When the results do not meet the standards of the evaluation, predefined interventions – like the Agent Kill Switch – are triggered. Where the team’s policy necessitates it, this will require approval.
Secondly, API Evals automatically create functional and performance tests from API specifications. This, Salesforce says, means breaking changes are caught before they ever reach production.
Thirdly, Salesforce is introducing advanced cost controls. The FinOps Dashboard and Model Wallet provide a view of AI spending with hard limits, which can stop requests before costs exceed budget. The company says that Cost Optimization Policies can cut token usage by up to 75%, based on test scenarios, through semantic caching, payload compression, and response trimming.
The company provides an example of a retailer that wants to cut the costs of a live order-support agent, so it switches its order-support agent to a lower-cost AI model.
Following the model change, Agent Evals will automatically run scheduled tests and discover that the agent is giving out inaccurate information about delayed orders.
The quality issue is then automatically flagged by Agent Evals, which requests approval to quarantine the agent – if there exists a preference to keep a human in the loop. If approved, the agent is then quarantined so the team can tweak the agent’s instructions on how to handle delayed orders.
They rerun the evaluation on the lower-cost model using the new instructions to confirm it’s performing as expected before returning it to production.
Evaluation and governance tools can help teams assess if AI agents are completing tasks correctly, using the right tools, operating efficiently, and following security policies.
Having those findings linked to predefined responses can cut down the manual work needed to manage agents across platforms and keep them working as intended after deployment.
Andrew Comstock, SVP & GM of MuleSoft & Agent Fabric, said: “Enterprises can only scale agents as fast as they can keep them accurate and on budget. Today, spotting a wrong answer or a spending spike is one job, and stopping it is another.
“Agent Fabric unifies them, bringing evaluations, cost controls, and intervention together so teams can contain issues instantly and scale agents with confidence.”
Poonam Deshpande, Director of Engineering, Digital Commerce at ASICS Digital, said: “What excites us most about Agent Evals is that it doesn’t stop at telling us something is wrong. When an agent’s quality drops, Agent Fabric can step in automatically, alerting the right owner or pulling the agent offline before a small issue becomes a widespread customer problem.
“That’s the missing piece for running agents at scale, and it will give us the confidence to keep expanding AI across our business. We are excited to partner with Salesforce and help shape Agent Evals Beta.”
Final Thoughts
Salesforce’s latest Agent Fabric additions tackle arguably one of the biggest challenges facing businesses which want to scale their AI agents – namely, keeping them reliable, secure, and cost-effective once they reach production.
The Agent Kill Switch seems to be the main takeaway, offering a method to move beyond simply identifying features and towards ever more automation to police your already existing automation. But, as is the case with so much automation, these evaluations will only be as useful as the tests and thresholds underpinning them.
