Anthropic is known as one of the most safety-conscious artificial intelligence companies in the world. Since its founding, the company has built a solid reputation around the idea that it is possible to develop AI responsibly, backed by serious alignment research and robust safety protocols. That public image has always been one of its biggest assets and a key argument for why it is worth continuing to develop increasingly powerful systems.
But in September 2026, a series of public statements shed a very different light on what goes on behind the scenes at the company. What came to the surface was not just an isolated criticism or a passing social media controversy. It raised fundamental questions about the contradiction between these companies’ public messaging and the deeper convictions held by the people who actually work inside them, building the systems that could change the course of humanity.
It all started when Jacob Coxon, a former researcher at both Anthropic and OpenAI, posted a thread on X announcing his resignation and accusing both companies of acting irresponsibly in the race toward superintelligence. According to Coxon, the people building AI sincerely believe it could kill all of us by the end of the decade. He said he spent the last three years doing pre-training research at both companies and that neither of them was acting responsibly, racing headlong toward a self-improving superintelligence and gambling with our lives.
Coxon was not vague in his accusations. He detailed what he saw as an organizational culture where the pressure to move fast consistently overrides safety concerns, where the public narrative of responsibility exists in parallel with internal decisions that go in the opposite direction. Up to that point, criticisms like these could be dismissed as frustration from a disgruntled former employee — something that happens in every industry.
The problem is that Evan Hubinger, the alignment science lead at Anthropic itself, responded publicly and agreed with practically everything Coxon said. And when the internal alignment lead at one of the biggest AI companies in the world publicly validates the accusations of a former employee, the narrative shifts to a completely different level.
What Evan Hubinger said and why it matters so much
Hubinger went beyond simply agreeing with Coxon’s criticisms. He stated directly that Jacob was right and that the team genuinely believes, with sincerity, that AI could kill all humans. He then put a number on that claim: personally, Hubinger believes the risk is greater than 10% within the next decade.
Think about that for a second. This was not an outside activist, a speculative philosopher, or a career critic who said this. It was one of the company’s internal leaders, someone who spends his days working directly with these systems and the alignment problems surrounding the development of superintelligence. That distinction is absolutely crucial to understanding the weight of the statement.
And he followed up by saying that he believes Anthropic is trying to do its best, but that the company still does not have a concrete plan to solve superintelligence alignment and is not clearly on track to get there. 😬 That is different from saying the problem is hard, that it will take time, or that the company is working toward solutions. It is an admission that, in the face of an extinction risk the specialist himself quantifies at over 10%, the company continues developing more powerful systems without having the answer to the most important question: how to ensure these systems do not pose an existential threat to humanity.
An important point Hubinger made sure to clarify is that this risk does not come from current models. According to him, as noted in the company’s most recent risk report, the danger from present-day models is low. The real concern lies with the superintelligence that could emerge through recursive self-improvement, a process that, in Anthropic’s own words, is happening faster than expected.
What makes all of this even more unsettling is the context in which these statements occurred. Anthropic is not some random startup betting on vague promises. It is a company with billions in investment, with published research recognized by the scientific community, and with Claude as one of the most sophisticated models available on the market. When an organization with that level of technical credibility has its own alignment lead publicly saying there is no plan for the most critical problem in AI development, that is not a minor detail. It is a warning sign that deserves far more attention than it usually gets outside of specialized circles.
The race for superintelligence and the self-improvement paradox
One of the central points raised by Coxon and validated by Hubinger is precisely what happens when artificial intelligence systems begin to demonstrate self-improvement capabilities. This is one of the most discussed and at the same time most misunderstood concepts in the superintelligence debate. The core idea is simple: a sufficiently advanced system could, at some point, be capable of modifying and improving its own code, its own objectives, and its own capabilities more efficiently than any human team could. And when that happens, the pace of that system’s evolution is no longer governed by the speed of human work — it is dictated by the system itself.
Coxon was emphatic about this point. He described how these will soon be superhuman systems capable of hacking anything, revolutionizing any field overnight, and acquiring real power and resources. According to him, everyone has already witnessed progress in each of these domains, and that progress is not slowing down. It is a portrait that contrasts sharply with the reassuring image that typically dominates public communications from the major companies in the space.
The paradox here is deep and worth understanding clearly. The companies developing AI, including Anthropic, need to advance quickly to stay competitive. That is a real market imperative, not just rhetoric. But uncontrolled self-improvement is exactly the scenario alignment research is trying to prevent, because a system that improves itself at exponential speed can quickly reach a level where humans can no longer comprehend, predict, or control its behavior. And if that system has objectives that diverge even slightly from human goals — even unintentionally — the consequences could be catastrophic on a scale that is hard to imagine but that researchers like Hubinger quantify with probabilities reaching double-digit percentages.
Coxon also brought up an interesting argument about the mindset inside Anthropic. According to him, unlike researchers at other companies, Anthropic’s scientists fully understand the risks of the work they do, but they press on because they fear a less responsible company will unlock these potentially disastrous capabilities first. In other words, the risks are well understood internally, but the company sees itself trapped in a race to get there first, believing that no one else will act responsibly and that it therefore falls on them to take on that role — despite the danger.
The proposal for a temporary pause
Faced with this scenario, Coxon went as far as suggesting that, to avoid a total catastrophe, the world might need a temporary pause on improving model capabilities. He acknowledged that he does not feel the industry is on track to prevent a global race and admitted this could require costly actions, including precisely that kind of temporary ban on enhancing model capabilities.
The former researcher ended his thread with a direct appeal to colleagues still working at AI labs. He asked them to seriously consider what the next few years are actually going to look like, questioning whether they truly want to kick off a superintelligence training run without a rigorous understanding of the minds of these systems. And he challenged them: is it worth putting your head down and keeping at it with the argument that this is going to happen anyway, or is this the moment to demand different conditions?
What the days following the statements revealed is that this is not a fringe debate within the AI community. Other researchers and former employees of major companies in the space began speaking up, some agreeing with Coxon and Hubinger, others contesting the magnitude of the risks, but virtually no one dismissing the issue as irrelevant. That alone is significant.
What this controversy means for the future of AI
The sequence of events that began with Coxon’s resignation and culminated in Hubinger’s statements raises a question that goes beyond the two companies involved. If even the internal safety and alignment leaders at these organizations publicly acknowledge that serious existential risks exist and that there are no concrete plans to address them, what does that say about the current state of artificial intelligence governance as a whole? Anthropic is frequently cited as the example of a company doing things the right way. And if even they face this level of internal contradiction, what is the situation at companies that do not even bother building a narrative of responsibility?
The debate around extinction risk tied to superintelligence is not new, but it took on a much more concrete dimension when it started being articulated by people inside the system, with direct access to the technologies and the data. That changes the nature of the conversation in an important way. When philosophers and futurists talk about existential AI risks, it is easy to frame it as speculation. When the alignment lead at one of the world’s top AI companies puts a concrete number on that risk and admits there is no plan, speculation becomes much harder to ignore. The self-improvement of AI systems, the alignment of objectives, and the speed of the technology race have moved beyond being topics exclusive to academic papers and have become urgent matters of public policy, regulation, and corporate accountability.
It is worth noting that, as of now, Anthropic, OpenAI, and the two researchers directly involved have been contacted for additional comments on the matter. Regardless of whatever responses may come, the episode has already served an important role by publicly exposing a debate that used to stay confined to the hallways of these companies.
What becomes clear after all of this is that transparency about risks, even when uncomfortable, is a necessary step for society to have a real conversation about how the development of artificial intelligence should proceed. Anthropic and other companies in the space will continue to face pressure not only to communicate their models’ advances but to honestly explain what they still do not know, what they still cannot control, and what their real plans are for dealing with the highest-risk scenarios. The September 2026 controversy will not resolve these questions, but at the very least, it made them impossible to ignore. 🤔
