Hello, everyone, and welcome to all newly joined subscribers!
This is the 90th edition of The Web3 + AI Daily - your definitive guide to the intersection of blockchain and AI. Today’s edition attempts to debunk some powerful myths surrounding AI, and to re-emphasize the importance of open-source software.
Thank you for being here! Let’s dive in.
What’s Hot in Web3 + AI?
Who Benefits from Claims that “AI Is Going to Kill Us All”?
It’s been a couple of weeks filled with troubling AI news. From OpenAI agents “going rogue” and hacking Hugging Face, through OpenAI agents “solving” one of the Millennium open math problems (I told you about it in the previous issue), to the former Anthropic AI Systems Developer Jacob Coxon warning that “self-improving AI models will end up killing us all.”
The wave of panic these statements have created is entirely understandable. But as I watch it unfold, I feel compelled to offer another perspective. There are plenty of researchers and scientists who consider these claims unreasonable, if not outright false, but their voices rarely reach the broader public. Fear makes for a much better headline than nuance, and terrifying predictions will almost always drown out cold reason.
So today, I’ll share some of the voices I follow, in the hope of restoring a little gumption where irrationality may have taken hold.
The “AI going rogue” narrative is one we’ve heard repeatedly over the past few years, but it truly took center stage amid the OpenAI - Hugging Face debacle. It aims to suggest that a superintelligence is emerging and machines are getting smarter, but a recent essay by Eryk Salvaggio directly challenged this. His main thesis: OpenAI turned off all of the model’s safety mechanisms, gave it impossible tasks and no way to quit, and left a door open for the model to access the internet. The results shouldn’t surprise us.
If you optimize a model to find exploits, you should expect it to find them — and prepare for that. OpenAI did not. They built a model, took the safeguards off, gave it the ExploitGym task, let it run, and didn’t even monitor it. That’s human decision-making.
Every time we read about AI and the race toward AGI, we should remember that an unbelievably enormous amount of money is being poured into this industry, and that almost no one involved is doing it out of the goodness of their hearts.
Both OpenAI and Anthropic began with missions centered on building AI that would benefit humanity. Yet both eventually moved toward commercializing their technology in order to attract the billions of dollars needed to compete in the race.
There’s another reason to approach the AI conversation with a healthy dose of skepticism: it is often contradictory, confusing, and deeply shaped by context. Depending on the setting and the moment, you can hear the very same person (Elon Musk, Sam Altman, Dario Amodei) argue that AI is the best thing that has ever happened to humanity, and that it could kill us all; that it is extraordinarily dangerous, and that it is indispensable; that AI should be strictly regulated, and that only certain players should be regulated.
I’m saying this to remind you that we should pay attention not only to what is being said, but also to who is saying it, when, and what incentives they have.
Timnit Gebru is someone I turn to every time I get confused. She has the ability to bring clarity and structure to media whirlwinds whose only goal seems to be generating hype. I can’t wait to read her new book and I strongly suggest you follow her, too.
Finally, I want to share a thread that, I hope, will calm you down, and perhaps even bring a smile to your face. It offers a striking illustration of just how absurd some doomer claims can be, and why they are often optimized for shock rather than for portraying realistic scenarios.
Let’s analyze some plausible ways this apparently most dangerous software, running in some servers (that can be damaged by throwing a bucket load of water at them) can wipe humanity off the face of the earth & why each point is highly unlikely (like maybe below 1e-30 chance).
Web3 + AI Readings & Conversations
Why Open Models and Open Science Matter
At the end of August, Erica Kang and NEAR AI hosted the 10th Open Source AI Summit (OSAS) in San Francisco. One of the most interesting conversations of the seminar was the one NEAR Protocol‘s Illia Polosukhin had with Matt White, former Global CTO of AI at The Linux Foundation and CTO of PyTorch.
When asked why open source software is important, White started by clarifying what “open source” actually means in AI. There are many open weights models, i.e., their code and weights publicly released, which have community licenses. This means that they have restrictions and acceptable use policies that can change arbitrarily. This creates confusion and frustration for people trying to build a business using these models.
What’s more, while model architectures and weights are often publicly accessible, research publications are actually shrinking.
At the same time, what “open source” should really signify is a model with a permissive or copyleft license, with which you could do everything you want to do.
“Generally, openness means that you have the ability to use, study, modify, and redistribute for any purpose.”
How to Create Better Incentives for Building Open Source?
It’s very difficult to find the money to build something and then release it for everyone to use for free. As White commented:
“There is no incentive mechanism that makes it okay that someone else will be replicating your work.”
On the contrary, the incentive right now is to build closed-source AI and strive for uniqueness, so you can raise funding for more compute. And this opens an even larger gap between needing giant private companies in order to fund research and independent scientists wanting to move the space forward.
Nevertheless, White confirmed that the open source movement is still going strong:
“There is still a big [open source] movement. I would love to see more of it happen in the US, and there be more open inter-lab competition, because I think that really spurs innovation.”
On the USA - China Competition
When asked about the AI rivalry between the USA and China, White explained that the paradigms in the two countries are very different. While in China open-source software is the de-facto, and there are many open-source labs aggressively competing with each other, this is not the case in the US.
China is also heavily resource-constraint on compute, and that scarcity forces them to innovate on model architectures, performance, and optimization. Their labs have been releasing a lot of technical reports and research papers, introducing new attention mechanisms and architecture variants, from which the US companies benefit as well.
“One trend I really appreciate is optimization engineering, where folks are spending time figuring out how to get more out of their compute. This is principled for many of the Chinese labs, whereas in the US, we’re like “Oh, compute is an endless resource, let’s just keep going and not focus on optimization.”
On the “AI is Too Dangerous Narrative”
Like Polosukhin, White refused to sign the open letter urging the U.S. government to intentionally slow down AI development - a stance that led them to discuss some of the doomer claims I outlined above. Here is White’s comment:
“First, it was the narrative that the most powerful models cannot be open source because they are too dangerous, it was not safe to put them in the hands of the public. Then, these models started hacking other systems, and the narrative was again “Oh, frontier AI is too dangerous”.
“It’s a sticky narrative, but it’s fundamentally flawed. The idea that the frontier can exploit systems is not only an offensive feature. It’s also very defensive. Hugging Face kind of demonstrated that with not being able to use ChatGPT or whatever model there were using with OpenAI at the time*. They were getting refusals to be able to inspect the logs and see what was happening. They were able to decipher what was happening by using GLM 5.2.”
(*When Hugging Face tried to use Anthropic and OpenAI models to defend itself, their guardrails turned up because the intended use was for cybersecurity purposes.)
“I think having that frontier level capability in open models gives the power to everybody to be able to create fixes and patches, test their networks, do pen testing, red teaming, etc., where otherwise you have to get in line. [...] Gatekeeping the ability to secure your systems through one or two labs is not really ideal.”
Polosukhin concluded by reaffirming the many ways in which blockchain and Web3 can help make AI safer and more reliable. The solutions exist, all the industry needs to do is start adopting them.
“A lot of the blockchain thinking revolves around the idea that there’s always someone trying to attack the system and how can the system withstand that. So, there are a lot of cryptographic methods to make sure there is a resilient ledger, so agents cannot rewrite their own logs or try to change something. There’s going to be a provenance of them trying to do that.”
Thank you for reading! I hope you found this article insightful.
If you’d like to keep up with the latest developments at the burgeoning intersection of blockchain and AI, I invite you to subscribe.
And if you find value in my writing, consider supporting the newsletter. I’d greatly appreciate it.
The Web3 + AI Book Club is live on Fable. Our September title is 'The Nerd Reich' by Gil Duran. Follow the link below and come join our literary community.
I'm looking forward to connecting with fellow Crypto x AI enthusiasts, so don't hesitate to share your thoughts here or reach out on LinkedIn, Instagram, or Firefly.social.
Disclaimer: The content of this newsletter is provided for informational and educational purposes only. Nothing contained herein should be construed as financial advice or as a recommendation to buy, sell, or hold any of the companies or assets mentioned.
Please don’t take my views at face value. Instead, do your own research (DYOR), think critically, and share your perspective so we can challenge ideas, learn from one another, and arrive at better conclusions through thoughtful discussion.