Highlights
- The AI regulation debate continues to be a hot topic within the tech space, with AI leaders like Dario Amodei, Sam Altman, and Marc Benioff all sharing their thoughts.
- As the rate of AI innovation continues to increase, deciding whether or not AI should be regulated becomes an even more pressing issue.
- There are different arguments for and against this regulation, and in this article, Sasha shares her argument for AI regulation whilst Tim shares his argument against it.
When former Anthropic employee and whistleblower Jacob Coxon took to X to share his thoughts on why he was leaving the AI giant, he spoke in detail about how Anthropic and OpenAI were “gambling with our lives” and that the people building the AI “earnestly believe that it could kill us all by the end of the decade.”
This isn’t the first time we have heard something like this about artificial intelligence or its makers, and it won’t be the last. However, it catalyzed a new wave of thinking about AI, specifically whether it needed regulating, and if so, how much. At Dreamforce, Salesforce’s flagship conference, AI regulation was a hot topic, and in a session with CEO Marc Benioff and the hosts of the All-In podcast, conversations reached a head. So, is AI really going to kill us, and does it need to be regulated?
A Possible AI Annihilation Scenario
Coxen’s initial post on X has now been viewed over 173M times. His seven-part thread, detailing how he perceived Anthropic and its competitors were approaching the future of AI, has also been viewed by several millions of people, garnering hundreds of thousands of responses.
“This is not a marketing stunt,” he wrote, in regard to the “AI will kill us all” statement. “If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”
He explained that at OpenAI, many have allegedly “not deeply internalized the civilizational stakes.” At Anthropic, the stakes are allegedly “well-understood,” but they are “locked in a race to get there first”, as they believe “no one else will act responsibly, so they must do it themselves, despite the risk.”
Glorified Autocomplete
Something that may not have been clear when listening to these conversations is remembering what modern AI is at its core. Large language models (LLMs) essentially work by predicting the next word in a sequence, just like autocomplete does. The difference is that it has a LOT more examples of what the next word could be, due to being trained on billions of parameters.
New technologies are often confusing and misunderstood as they slowly become more and more accepted as mainstream. We’ve seen this with the internet, for example. Many believed that it would be a passing fad (note the similar terminology: the “AI bubble”), or a glorified fax machine.
Over time, as people began to use these technologies more and more, they became more “normalized”. They were no longer mystical or magical, but rather just a very useful tool that had been adapted to our needs.
When it comes to the AI era, we’re still in the relatively early stages. There is a lot of fear, uncertainty, and doubt surrounding it. There are many who flat-out refuse to touch AI and are fearful of their content being used by AI in negative ways.
These concerns are absolutely valid, especially when you consider how quickly AI is being adopted by seemingly every technology company. However, we’ve noticed that sometimes this is due to a lack of understanding of exactly what AI is and how it works at its core.
Where Do AI Leaders Actually Stand?
Earlier this month, Anthropic CEO Dario Amodei shared an open letter titled “We Must Pace the Frontier”, detailing how the scale of AI’s innovation needed to be reconsidered. He explained that although AI has the potential to solve a variety of relevant, real-world problems, the risks were becoming too serious to continue at the current pace of innovation.
“We must slow the pace at which we improve the capabilities of AI models,” he wrote. “Progress will still seem fast, and we must make wise use of the time we gain.”
Initially, other AI leaders came out in support of Amodei’s message, including OpenAI CEO Sam Altman and SpaceX CEO Elon Musk. Altman also spoke in a session at Dreamforce, saying he believed that safety and monitoring should take precedence over features when it comes to AI development, whilst also acknowledging the growing hesitance to believe that AI companies will do the right thing.
However, in that same session, he also insinuated that AI companies were creating fear around the technology through pushes for change, and that “the world should trust that we are going to do the right thing because it’s the right thing.”
On the other hand, tech leaders like Mark Zuckerberg and Jensen Huang have also come out with their opinions on the matter. Huang said in the Dreamforce keynote that companies should be allowed to “run as fast as they can” and that the “most important thing [is to not] get left behind”.
What About Benioff?
Initially, it seemed that Marc Benioff’s opinions sat somewhere in the middle; AI companies should be able to decide their own pace of innovation, but they should also “be held accountable” for their decisions.
However, Benioff, who looked worn out and unwell at the time, reopened the conversation with the hosts of the All-In podcast on one of the last Dreamforce stages.
The Regulation Conversation
Interestingly, David Sacks, one of the podcast’s hosts who attended the session virtually, detailed how he believed there are three ways to approach AI regulation.
“Number one is we can pause,” he said. “We can just stop. If we do that, I think we all understand that won’t actually solve any problems – it’ll just hand the frontier to China.”
“Option number two is we send in the regulators, and they make all the decisions about what gets released or not. The problem with that is […] the space is moving too fast, and the regulators don’t have the technical capabilities to do that. “
“So that brings us to number three, which is that the companies have to internalize a sense of responsibility, but the government creates the incentive structure.”
As the conversation continued, Benioff compared this particular AI problem to Gutenberg and the printing press, and how there was an effort to control that technology through the Vatican.
“If you go back to that moment and ask yourself, would we have been in a better moment?” he asked his fellow panelists and the audience. “Do you think it would have been better if it was controlled by a singular organizational body? No.”
“I just think the artifacts tell us diffusion works, control how you deal with the aftermath, do root cause analysis, be profoundly serious engineering organizations, and things will be okay.”
David Friedberg, another All-In host, shared his perspective on the matter, saying that Big Tech has “convinced the public that they’re gonna lose their jobs and get killed by Terminator robots, which has not been great for the industry.”
Chamath Palihapitiya, another host, added that it’s difficult for companies, when they’re growing, to slow down because “you’re not rewarded for slowing down”, shedding light on the uncomfortable truth of this conversation.
“I was at a dinner party last night,” Sacks said. “Obviously the topic of the day was, you know, AI, whether it could end humanity. I realized, you know what – my side of the argument is this is way too boring. It’s a lot more fun to talk about the Terminator and recursive self-improvement.”
To Regulate or Not to Regulate
The regulation conversation – even with multiple data points and opinions – still doesn’t mean that everyone has collectively decided on whether or not AI should be regulated. In fact, we’re torn here at SF Ben, which is why we decided it was important to share those opposing views with you in one place.
For Regulation: Sasha
Marc Benioff himself has often compared the AI regulation situation to the mistreatment of appropriate safeguarding with social media. “We must hold companies and technology executives responsible for the technology they build,” he said in a press session I attended last week. “I hope that we will take those lessons and say that social media is probably just a warm-up for AI.”
I agree wholly that we should use the case of social media as a learning curve for what we’re currently experiencing with AI. For some parts of the world, meaningful social media regulation is only just beginning to be brought in. A lot of it is being scrutinized, especially here in the UK, but ultimately, it was decided that something needed to be done to safeguard its users – many of whom are young, disadvantaged, and impressionable. AI appeals to these groups of people too – and we see it materialize in the most unfortunate ways through documented cases of harm.
On an enterprise level, I believe regulation is needed across the board because we just can’t trust that these AI companies will do the right thing, even if people like Sam Altman want you to believe that they will. I mean this regarding the obvious facets like AGI, but also concerning the environment, job displacement, and control over such a powerful technology.
That is a lot of responsibility to hand to such a small group of people. Sam Altman even said one of his biggest fears was “AI authoritarianism”, and warned that humanity could go “very badly” if “a small number of people, companies or countries accumulate enough AI-driven power to impose their will on others.” Leaving it to a governing body will have its own challenges, but without it, the challenges we could face would likely be unprecedented.
Against Regulation: Tim
While I agree with Sasha that things need to change, I’m a big believer in personal responsibility rather than government intervention. Regarding the example that Sasha used of social media regulation, for example, I think the current approach where governments are trying to regulate social media isn’t working.
My own country, Australia, has seen a significant backlash following ever-increasing government meddling in our lives, and this move in the form of personal digital regulation is no different. Instead, I personally think a better approach would be to give users more power and control over themselves. Allow parents to set up and monitor their children’s social media access, for example, rather than allowing government access to more and more of our personal lives.
Looking back at AI, I think it’s a bit trickier given the scale of this new era of technology. It’s not as simple as algorithms showing developing minds new ideas or content that may be inappropriate. We’re talking about technology that can be used to create content at a scale and speed that’s never been seen before, and as such, we can’t predict just how it could/would be used.
So, in theory, it does need to be guided properly, but is government regulation the right way to do it? I don’t believe this is the case. However, I also unfortunately cannot propose a more effective way to do it.
My concern with government regulation is exactly what Sasha mentioned above: Sam Altman has discussed his fear of “AI authoritarianism”, and my belief is that governments of major countries represent that “small number of people” that could, or perhaps would, use that power to impose their will on their population.
Do I think that AI is going to kill us all? No, not on its own. AI is fantastic, but at its core it’s a glorified autocomplete. It’s an algorithm. It’s mathematics. Mathematics alone has never killed anyone, and in fact it has been used to do some wonderful things as well. It drives medicine and medical research to the point where many illnesses that ended lives earlier in the past are now treatable at home. We’ve got the ability to harness the power of the sun and wind to power our homes and fuel our cars. All this comes from science and from mathematics.
However, these things have been used to create some horrific weapons that have caused devastation across the world. Weapons have caused devastation and been used to control both masses and individuals. While I flagged earlier that we’ve grown as a species as a result of mathematics, there’s no doubt that we’ve also caused unimaginable suffering too.
So, should mathematics be banned, regulated, or feared? No, but some of its deployments absolutely need to be guided properly, and there are some deployments that should be guided by various governments.
Coming back to the original question, no, AI should not be regulated. That being said, there are some deployments of AI that do need to be watched and carefully guided. Whether that’s by governments or by the companies and groups deploying the technology,
I’m not sure. We should be very careful rushing to call in regulation, though, or we could end up with innovative technologies being stifled by red tape and bureaucracy.
Final Thoughts
The Dreamforce session with Marc Benioff and the All In team was an insightful discussion on the future of AI and its impact on society. There are absolutely concerns, many of which are valid, but there’s also quite a bit of empty rhetoric. Time will tell whether AI in and of itself poses a serious threat to humanity, whether government intervention is required, or whether vendors really do have the ability to safely control its future.
Perhaps David Sacks is right: there are three approaches to ensuring the future of AI is safe and secure: halting development, handing over regulatory responsibility to the government, or letting the companies and groups deploying the technology take responsibility, with government driving incentives. Which of these is best, and which actually plays out, is yet to be seen.
