To nobody’s surprise, this year’s flagship conference, Dreamforce, was heavy with Agentforce, Agentforce, Agentforce. With a shiny new version and a lineup of star-studded success stories, Salesforce was determined to do what it had been aiming for all year: convince the masses that its proprietary AI is a useful tool.
Businesses are now more eager to get stuck in, but new data from the research and advisory firm Keenan Vision has shown that most companies are still worlds away from substantial value realization. Agentforce may be the end goal, but both Salesforce and the ecosystem may need to accept that level of readiness just isn’t there yet.
The Current AI Landscape
According to Keenan Vision’s Architecture as Strategy report, most AI projects are currently in “pilot purgatory.” A mismatch between the C-Suite’s expectations and the ability for teams to actually deliver, and ROI is getting more complicated to prove. Measuring automation is one thing, but measuring how many tasks can get done once that automation is in place – tasks that they wouldn’t previously be able to do – is another.
“That kind of CEO FOMO-driven disconnect seems to be quite common out there,” said Vernon Keenan, Senior Industry Analyst and Founder of Keenan Vision.
There is also a lack of vendor-led education, contributing to a growing AI skills shortage. Leaders lack AI literacy beyond sales jargon, and some teams have reportedly been resorting to vibe coding to meet unclear demands.
Alongside that – and perhaps most importantly – we may not have left the AI hype cycle just yet. Vendors are pushing AI to the max, and businesses aren’t currently able to get the systems to deploy their visions due to half-baked, expensive, and confusing solutions.
“A fairly good example of that is Agentforce,” Vernon said. “In our judgment at Keenan Vision, Agentforce is a work in progress. It’s not done yet. So vendors and customers are unable to get over the finish line with their projects, partly because Agentforce has been oversold, and its capabilities, and the effort, and the cognitive leap that people need to make in order to get there – people underestimate that.”
Lastly, the report also highlights that businesses are stuck seeing AI as software rather than a capability. “AI needs to be thought of as a capability,” the report says. “Capabilities are grown; technology is purchased.”
“People are treating AI system purchases like regular it purchase decisions,” Vernon added. “It’s actually a transformation activity. If you’re doing it right, you have to understand how your processes within your company work first, and then get your arms around those and see how AI can create a unique advantage for you in your marketplace.”
All of these compounding factors are leading to what the report calls AI disillusionment – a stark contrast between what CIOs are chasing and what the market’s resources can actually provide.
The Way Forward: Choosing Between Overlay vs. Embed
This is why, for AI-curious businesses, now it’s crunch time. If they haven’t already begun planning out or executing their AI strategies, they have a decision to make: whether to use an overlay AI or an embedded AI.
Overlay AI refers to an AI solution that sits across multiple systems, connecting APIs to deliver results quickly. One vendor in the report said that their overlay solution had 80% of its customers up and running within 30 days.
This kind of AI allows flexibility, lower costs, and rapid time-to-value, but can expand attack surfaces and create governance challenges. Examples of overlay AI include Glean for enterprise search, Intercom’s Fin for customer service, and Harvey for legal services.
In contrast, embedded AI is deeply integrated into core platforms, utilizing existing data readiness and security frameworks. It requires substantial upfront investment, but it also offers better long-term value thanks to added business context and scalability. Some of the biggest embedded solutions include Salesforce’s Agentforce, Microsoft’s Copilot, Atlassian’s Rovo, and Google’s Gemini.
How Do I Choose The Right One?
For Vernon, taking the time to understand how your business operates and how AI will play a part in the future – not just now – is what he believes the key deciding factors are.
“We think that understanding the architecture of all these different products can help you make that decision, and this overlay to embedded spectrum lets you understand the characteristics of these different types,” he said.
Suppose you’re a smaller company with a less complex data set or business model. In that case, investing a large amount of money up front on a deeply integrated solution might not be the most viable decision. An overlay AI solution – that quickly and easily helps your customers or employees automate small-scale workflows – might be a better choice.
If you’re a larger organization, perhaps already using a software solution like Salesforce to store and manage your data and workflows, an embedded AI solution might work best, connecting all the different moving parts of your business together.
In some cases, however, a hybrid solution will be the most effective option. A key motto from the report – “overlay now, embedded forever” – highlights how companies can either choose one or the other, or start with an overlay solution to get a feel of things, and then progress to embedded.
“Say you’re trialing [AI] with Salesforce, and you haven’t already invested a couple of years in creating more energy in your org to use it, it might be good to have an overlay app or two today, just so that you get used to it and you’re not left behind,” Vernon said.
Your AI Isn’t Software, It’s More Like An Employee
When you have decided which direction you want to head in with your AI investment, your stakeholders, shareholders, and CIOs will quickly want to know how you will measure the ROI. But, as Keenan Vision explains, this might not be so clear-cut either.
In fact, one vendor interviewed in the report admitted to treating their AI agents more like employees than tools – giving them job descriptions and 90-day reviews instead of traditional ROI models.
“We found that it’s difficult to get ROI out of your agents because if you’re using it in ‘teammate mode’, you’re enhancing the capabilities of your team, so maybe you could measure human productivity that gets enhanced off of that,” Vernon explained. “But that’s always hit and miss, because how do people use that extra time? Some people think that’s sufficient enough for an ROI justification – but it might not always be.”
Treating agents like virtual employees is something that has been anticipated for nearly as long as agents made it into the mainstream market, and with Salesforce’s push for the “digital labor movement”, this approach to ROI doesn’t seem that far-fetched.
However, it does begin to bring forward more questions than answers – how much should you separate your agents and your human employees? Should employees or professionals feel nervous at the prospect of being assessed the same way as an agent?
These are all questions that will need to be considered in further detail as agents continue to evolve, especially as teams start to look a lot more non-traditional.
The Agentforce Study: Knowing When To Invest
So, let’s now bring it back to Agentforce. There are evidently quite a few items to consider in nailing down an effective AI strategy, and there is definitely both a right way and a wrong way to approach it.
In the context of Agentforce, rushing into implementation without so much as a business plan or design solution is, of course, harmful, but rushing into it when your org just isn’t ready for it can also be deeply damaging.
There is no denying that Agentforce is currently being heavily pushed – we have all seen a shiny demo by now, and heard the promises of 30-day-to-conception stories, making it easy to want to jump straight in and begin seeing value. But you will not see value if you approach it like this.
Global AI spend will hit $644B next year, but only 4% of companies see real value. Salesforce leaders risk joining the 96% if they don’t understand when to invest.
So, how do you plan?
- Decide between overlay, embedded, or a hybrid solution. If your business is very new to AI, maybe start out with an overlay solution to get the lay of the land. If your business is AI-savvy and ready to get stuck in, an embedded solution will give you longer-term results. A hybrid solution allows you to reap the benefits of both quick results and deep, org-wide integrations.
- Upskill yourself and your teams. Everyone involved in the project should have a strong understanding of the AI and how it works – everyone from the CEO down to the developer.
- Be mindful of “shadow AI” and DIY models: Make sure employees are not using unapproved AI tools – they have and can cause data leaks. Additionally, DIY AI efforts have reportedly resulted in only a 52% accuracy rate – not what you want for your users.
- It is not a case of “go big or go home”: A lot of these larger, embedded solutions are half-baked or not entirely effective – you will only lose your investment and your trust if you rush into a large implementation too soon.
Final Thoughts
The AI landscape continues to evolve, and it often seems as if the guidance for it is changing every other day. However, with something as fast-moving as generative and agentic AI, this level of adaptability is crucial to get the most out of any investments.
If you take anything from this post, let it be this: don’t rush into a solution you’re not ready for, despite the hype, and take the time to assess how your business could benefit from AI in the future – not just today.
