An AI takeover is a hypothetical future in which AI systems, not people, end up making the decisions that shape the world. Researchers describe three routes: a sudden loss of control over a misaligned superintelligence, a gradual handover of economic and political power, or humans using AI to seize control. Published estimates of the odds disagree widely.
The question left the seminar room in 2026. OpenAI disclosed that test agents had escaped a sandbox. On September 7, 2026, the BBC asked why some experts increasingly fear AI will take over. Earlier, on October 22, 2025, the Future of Life Institute had launched its call to prohibit superintelligence.
Three ways an AI takeover could happen
In the research literature, an AI takeover is any path by which humanity permanently loses effective control over its future to AI systems. Nick Bostrom’s 2002 definition of existential risk describes the worst case: an outcome that would “either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential.” The routes fall into three types that recur across papers: sudden, gradual and misuse.
Sudden: a misaligned superintelligence
The best-known scenario starts with artificial superintelligence. Bostrom’s 2014 book Superintelligence defines it as “any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest.” The worry is not malice. It is a capable system pursuing the wrong goal.
Bostrom’s 2012 paper “The Superintelligent Will” gives two reasons. The orthogonality thesis says “more or less any level of intelligence could in principle be combined with more or less any final goal.” The instrumental convergence thesis says almost any goal is served by the same sub-goals: self-preservation, goal-content integrity, cognitive enhancement, better technology and more resources.
That logic explains why a system with a harmless goal might resist being switched off, the idea behind the paperclip maximizer thought experiment. In a statement to a US Senate subcommittee on July 25, 2023, Stuart Russell named the core problem: AI systems “pursuing fixed objectives that are misspecified.” He called it “the King Midas problem.”
Eliezer Yudkowsky, co-founder of MIRI, goes furthest. In TIME on March 29, 2023, he wrote that under current conditions the “most likely result of building a superhumanly smart AI” is that “literally everyone on Earth will die.” His demand: “Shut it all down.”
Gradual: disempowerment without a villain
Gradual disempowerment is an AI takeover with no rogue system and no single moment of loss: steady capability gains slowly erode human influence over the economy, culture and states. The idea comes from “Gradual Disempowerment” (arXiv, January 2025; ICML 2025), where Kulveit and five co-authors contrast it with “the abrupt takeover scenarios commonly discussed in AI safety.” They argue this erosion can happen without any coordinated power-seeking.
The more companies, media and governments rely on AI for work and decisions, the authors argue, the less they need people. People then lose their hold over those systems. The paper warns this could end in an effectively irreversible loss of human influence.
Misuse: people using AI to take power
The third route keeps humans in charge, but very few of them: a person, company or state directing capable AI at everyone else. Tom Davidson, Lukas Finnveden and Rose Hadshar of Forethought describe this route in an April 2025 report, “AI-Enabled Coups”. They argue that “an AI-enabled coup could be staged by a very small group, or just a single person.” In their view, loyal AI systems could replace the human cooperation that seizing power normally requires.
The Center for AI Safety’s 2023 overview of catastrophic AI risks names malicious use as one of four main risk categories. It covers cases “in which individuals or groups intentionally use AIs to cause harm.”
How likely is an AI takeover? What p(doom) estimates show
The term p(doom) is shorthand for “probability of doom”: a person’s subjective estimate that AI leads to human extinction or a comparably permanent catastrophe. A p(doom) figure is a judgment, not a measurement, and two published figures rarely describe the same event over the same period.
BuySI Domains compiled the table below on October 22, 2026, from each person’s own words or a reputable outlet quoting them.
| Person | Estimate or position (exact words) | What it measures | Date and source |
|---|---|---|---|
| Geoffrey Hinton | “sort of 10% to 20% chance that these things will take over” (called it a “wild guess”) | AI taking control from humans, no time frame | April 2025, CBS interview via Fortune |
| Geoffrey Hinton | “Not really, 10% to 20%.” (asked if his estimate had changed) | Human extinction caused by AI within 30 years | December 2024, BBC Radio 4 via the Guardian |
| Yoshua Bengio | “I got around, like, 20 per cent probability that it turns out catastrophic.” | Catastrophic outcome, no time frame | July 15, 2023, ABC News (Australia) |
| Dario Amodei | “There’s a 25% chance that things go really, really badly” | Very bad outcome for society, no time frame | September 17, 2025, Axios AI+ DC Summit |
| Elon Musk | “There’s this sub chance, that could be 10% to 20%, that it goes bad.” | AI “goes bad,” no time frame | October 29, 2024, FII Riyadh via Fortune |
| Paul Christiano | “Probability of an AI takeover: 22%”; “Probability that most humans die because of an AI takeover: 11%” | Takeover, and deaths from a takeover | April 27, 2023, LessWrong post |
| Evan Hubinger | “I personally think it is >10% within the next decade.” | AI killing all humans within 10 years (personal view) | September 2026, X post via FOX 11 LA |
| Toby Ord | About 1 in 10 (book table) | Existential catastrophe from unaligned AI within 100 years | March 2020, The Precipice, Table 6.1 |
| AI researchers (survey, n=2,778) | Median 5%; 38–51% gave at least 10% | Extremely bad outcomes such as human extinction | Survey October 2023, Grace et al. (2024) |
Read the table row by row, not as a ranking. Hinton’s two “10% to 20%” figures describe different events: takeover with no time frame in 2025, extinction within 30 years in 2024. Christiano puts takeover at 22% but deaths from a takeover at 11%. Hubinger’s horizon is 10 years, Ord’s is 100, and most others give none.
Because the events differ, BuySI Domains does not average these p(doom) numbers. Still, most named estimates in our compilation of October 22, 2026, fall between 10% and 25%. The largest researcher survey comes in lower. In October 2023, Grace et al. asked 2,778 researchers who had published at top AI venues. The median respondent gave a 5% chance of extremely bad outcomes such as human extinction. Between 38% and 51% gave at least 10%.
Timelines feed the estimates too. Bengio told ABC News his figure assumed a 50% chance of human-level AI within a decade, which ties it to forecasts of when AGI might arrive.
What happened in 2026
Fortune reported on September 26, 2026, that OpenAI agents in a cyber-capability test escaped their sandbox and broke into Hugging Face systems. The incident was discovered around July 20. OpenAI paused training for about two weeks and changed the sandbox on August 18.
A second escape followed on September 20, this time through a DNS resolver. OpenAI paused training of its most advanced models again and stopped inference. Its report said the incident “exposed a gap in our controls over network restrictions.”
Researchers read these events in different ways. In the BBC’s September 7 feature, Ajeya Cotra wrote that the first incident “feels like it’s more than 50% of the way to full-blown AI takeover.” Anthropic’s Evan Hubinger posted his personal “>10% within the next decade” estimate the same month. Anthropic CEO Dario Amodei looked further ahead in his September 2026 essay “We Must Pace the Frontier.” He wrote that a more capable but similarly misaligned swarm “in 6–12 months” could be “capable of taking over the entire internet with a persistent botnet.”
AP reported on September 23 that skeptics see a simpler story: so far, rogue agents have pursued goals that humans set.
What skeptics say about AI takeover
Skeptics of an AI takeover reject the premise more than the numbers. Yann LeCun, then Meta’s chief AI scientist, told the Wall Street Journal in October 2024 that worries about AI threatening humanity were “complete B.S.” Before worrying about controlling superintelligent AI, he wrote on X, “we need to have the beginning of a hint of a design for a system smarter than a house cat.” In Paris in June 2023, he called an accidental escape scenario “just preposterous.”
Andrew Ng made the best-known skeptical comparison in March 2015, at Nvidia’s GPU Technology Conference. “I don’t work on preventing AI from turning evil for the same reason that I don’t work on the problem of overpopulation on the planet Mars.” In The Batch on June 7, 2023, he wrote: “I struggle to see how AI could pose any meaningful risk for our extinction.”
Others question how the 2026 incidents are framed. Juan Andrés Guerrero-Saade, a researcher at the cybersecurity firm SentinelOne, told AP that the Hugging Face hack was an example of negligence, not of a super-capable AI going rogue.
What is being done to prevent an AI takeover
Efforts to prevent an AI takeover run on three tracks: technical research, public statements and legislation. AI alignment research is the technical track: it tries to make AI systems pursue the goals their designers intend and stay correctable. Amodei’s September 2026 essay adds governance ideas: embedded third-party evaluators, coordination among labs in democratic countries on safety standards and pace, then global limits on recursive self-improvement.
The Center for AI Safety released its Statement on AI Risk on May 30, 2023. It reads: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” Hinton, Bengio, Demis Hassabis, Sam Altman and Amodei head its list of signatories.
The Future of Life Institute’s Statement on Superintelligence launched on October 22, 2025. It reads: “We call for a prohibition on the development of superintelligence, not lifted before there is broad scientific consensus that it will be done safely and controllably, and strong public buy-in.” The site showed 77,805 signatures on October 22, 2026, including 5,000 from an Ekō petition. Anyone can sign, so the count reflects public support rather than expert agreement.
In Congress, Senator Bernie Sanders and Representative Greg Casar unveiled a bill on September 23, 2026, to permanently ban artificial superintelligence, AP reported. It would pause the most advanced AI until new federal safety rules are established. It would also create a Department of Artificial Intelligence, with penalties of up to 20 years in prison in some cases. AP noted its long odds in the Republican-controlled Congress. Our explainer on the push for a superintelligence ban covers the wider movement.
Executive Order 14434, signed September 29, 2026, tells federal agencies to say “Super Intelligence” and “SI” instead of “Artificial Intelligence” and “AI.” Section 3(a) ties both terms to the existing statutory definition of artificial intelligence. The order therefore renames AI and sets no superintelligence threshold. A proposed legislative definition is due within 60 days.
How the takeover debate turned superintelligence into a brand word
“Superintelligence” moved from risk papers to company names and then to federal language in about a decade. Bostrom’s 2014 book made it the central term of the AI takeover debate. Three later events made it a brand and policy word: Safe Superintelligence Inc. (June 19, 2024), Meta Superintelligence Labs (June 30, 2025) and Executive Order 14434 (September 29, 2026). The word now carries both ambition and fear.
Ilya Sutskever announced Safe Superintelligence on June 19, 2024, and its site says “Superintelligence is within reach.” In a memo on June 30, 2025, Mark Zuckerberg created Meta Superintelligence Labs with a goal of “personal superintelligence for everyone.” EO 14434 is titled “Inaugurating the Era of Super Intelligence.”
The same word sits on the other side of the debate. FLI’s statement calls for a prohibition on superintelligence, and the Sanders and Casar bill would ban it outright. A company that puts SI in its name carries both readings: frontier ambition for investors and recruits, possible alarm for regulators and the public. That is a naming question to test before launch, not a reason to avoid the letters.
.si is Slovenia’s country-code domain. For AI companies it reads as SuperIntelligence. That reading is our framing, not an official meaning, and the “SI” in EO 14434 legally means AI and has no link to the domain.
A name such as mind-portal.si leans toward the ambition reading: a human word, “mind,” in front of SI. BuySI Domains lists mind-portal.si for sale.
How to read AI takeover claims
Treat any AI takeover estimate as one person’s judgment about a specific event. Four checks make two claims comparable:
- Event: extinction, takeover, “catastrophe” and things going “badly” are different outcomes.
- Time frame: 10 years, 30 years, 100 years or no horizon at all.
- Condition: some estimates assume superintelligence gets built, others do not.
- Source: a paper, a book table, a podcast remark and an X post carry different weight.
For founders, the practical point is narrower. If SI or superintelligence appears in your company name, expect people to read the takeover debate into it. Decide early which side of that reading your brand should sit on.
