Home / Questions

Existential Safety
Provocative Questions

See a downloadable Google Document version of these questions.

Risks

●

What would Adolf Hitler do with a jailbroken ChatGPT-X?

●

What would Adolf Hitler do with a jailbroken ChatGPT-X?

●

What would Adolf Hitler do with 100 million AI slaughterbots?

●

What would Adolf Hitler do with 100 million superintelligent slaughterbots? (He would only need one to take over the world. It would self-replicate as needed.)

●

If Adolf Hitler were alive today, would he prefer to have 100 Tsar nuclear bombs or one superintelligent AI?

●

What kind of harm could the world’s ~248M online individuals with psychopathy create with a jailbroken ChatGPT-X? If 1% chose to commit harm, how could we plausibly stop all ~2.5M of them?

●

How can we ensure an advanced AI is never jailbroken when billions of people will have an extraordinary incentive to do so (e.g., perceived immense money, fame, power, longevity, safety, etc.)?

●

For the first time in history there can now be millions of AI-augmented bioterrorists and reckless biological experimenters. What specifically can the UN, major superpowers, leading AI companies or the general public do to prevent a mass casualty incident from a bioterrorist using jailbroken frontier AI models? Or from a lab leak? When will they do it? Can they be 100% successful at their attempts for the indefinite future?

●

What would you personally do with 100 million superintelligent agents working 24/7/365 on your behalf? Would your army's actions be of benefit to all life on Earth? Would any of Earth's millions of species potentially be worse off? Can you know for sure what you ought to do or not do with this godlike power?

●

If you had 100 million superintelligent agents working for you, could anyone in the world stop you from enacting your desired changes? Could anyone stop your neighbor from enacting their desired changes? Or Ted Kaczynski enacting his desired changes? Or Osama bin Laden? Or a yet-to-be-discovered terrorist leader? Would wars between swarms of superintelligent agents fighting for different values be desirable?

●

Would you rather have an organic alien superintelligence or digital alien superintelligence in your backyard?

●

If digital aliens from Alpha Centauri beamed into our computing systems tomorrow, how would humanity control them or our computing systems? How could we stop the Alpha Centaurians from doing anything they wanted? Does your answer change if you replace Alpha Centaurian with a sentient, autonomously acting ChatGPT-X?

●

Do you think the leaders of the AI safety field–including Nobel laureates and others that actually invented much of the AI we’re now trying to understand–are wrong about the risks? If so, why are they wrong and you are right?

●

Do you similarly disagree with international nuclear safety agencies made up of nuclear safety experts who set safety standards for nuclear power plants? Or with standard settings bodies made up of experienced engineers who set safety standards for bridge design? Or with international health agencies made up of experienced medical directors who set safety standards for brain surgery procedures?

●

Are you so confident that there aren’t real risks or they can be easily and quickly mitigated that you’re willing to risk your life, your family’s lives, everyone else’s lives, all non-human lives, and all potential future lives on you being entirely right?

Prioritization

●

Can you achieve anything else you value if humanity destroys itself with AI in the next few months or years?

●

What is more valuable to you than trying to mitigate existential or catastrophic risk from AI now? Or if not AI, then from nuclear holocaust or an engineered pandemic?

●

Where does mitigating existential or catastrophic risks from AI rank on your list of daily priorities? (Most people never realized they could make a meaningful difference, so never took much action. You can always start now.)

●

What exactly would you need to see to make mitigating existential or catastrophic risks from AI the number one daily priority for you?

●

If you were an adult during the 1940s when nuclear apocalypse was becoming an unmistakable risk, what exactly would you have needed to see to make mitigating catastrophic risks from nuclear weapons the number one daily priority for you?

●

Do you consider the 33-50% odds of a nuclear holocaust during the 1962 Cuban Missile Crisis—according to United States President John F. Kennedy—was an acceptable risk for humanity to have borne for the benefits of having nuclear weapons available?

●

What would you have done to help prevent the nuclear holocaust that almost occurred during the Cuban Missile Crisis, if you had the opportunity? Or what would you have done to help prevent any of the dozens of other nuclear close calls that have occurred over the last few decades? Are you willing to help prevent future nuclear catastrophes through your time, dollars or votes now?

●

Would you personally have done anything to prevent that 1962 near holocaust from occurring if you were alive at the time? Or would you have helped prevent any of the dozens of others that have occurred over the decades?

●

How appreciative are you for the nuclear safety experts, policymakers, philanthropists, civil society organization leaders, and citizen advocates who worked to ensure you were able to be alive today? Have you reached out to one to thank them? Or donate, if you had discretionary funding? Will you consider doing so now?

●

Do you have a written COVID-2X plan, complete with equipment, supplies, and step-by-step actions you will take when a major pandemic hits again? If not, will you write one now? For reference, prediction markets estimate ~31% of another pandemic by 2030 and ~74% chance by 2040.

●

Do you have a written COVID-2X plan, complete with equipment, supplies, and step-by-step actions you will take when a major pandemic hits again? If no, why not? Step-by-step guides have been online well before COVID-19. For reference, prediction markets estimate ~31% of another pandemic by 2030 and ~74% chance by 2040.

●

What exactly would you need to see in order to begin seriously advocating for pandemic preparedness and allocating at least some of your discretionary time and money to prevent a COVID-2X or similar? Was COVID-19's ~27 million excess deaths and trillions in economic damage insufficient justification, given your worldview and other priorities?

●

What exactly would you need to see in order to begin seriously advocating for pandemic preparedness and allocating at least some of your discretionary time and money to prevent a COVID-2X or similar? Was COVID-19's 27 million excess deaths and trillions in economic damage insufficient justification for you, given your worldview?

Governance

●

We’ve known about the devastation that machine intelligence would likely inflict on humanity since 1863. In 1951, amidst the digital computer revolution, Alan Turing cogently predicted this risk. The United Nations recognized these risks in 2023, but neither they nor any nation has yet to declare mitigating these risks as their number one priority. Why has our leadership done so little to prevent these clear risks?

●

We’ve known about the devastation that climate change would likely inflict on humanity since 1896. Why has our leadership done so little to prevent these clear risks?

●

We’ve known about the devastation that nuclear weapons would likely inflict on humanity since 1933. There are still 12,000+ nuclear weapons in the world today, including many that are literally missing. Why has our leadership done so little to prevent these clear risks?

●

We’ve known about the devastation that pandemics cause on humanity since before 430 B.C. Why has our leadership done so little to prevent these clear risks?

●

If a planet killer-sized asteroid were on a collision course with Earth within the same timeframe that is expected with artificial general intelligence—roughly one to six years according to some estimates—what odds would you put on humanity developing the geopolitical coordination and technological know-how necessary to neutralize the asteroid in time?

●

What level of extinction risk should a new commercial product be allowed to impose on humanity, if any?

●

What level of extinction risk should a new commercial product be allowed to impose on humanity? Any?

●

Should any person or company be able to control an army of millions of robots? Is there any viable way to stop a person or company from building this army given the continual decline of manufacturing costs and increase in access to production designs?

●

Should any person or company be able to unilaterally administer an experimental, dangerous serum on billions of people without their informed consent? What about an experimental, dangerous digital serum like AI?

●

The founders of frontier AI companies have known about the existential risks their AIs would pose to humanity since before they invented them. The "AI suicide race" that we're now in has been predicted since at least 1965. Why did AI leaders not use their time, talent, and money to prevent these risks by building or improving AI safety-focused global governance organizations before leading existentially risky research into frontier AI? For example, by donating to the United Nations for AI safety? For reference, in 1997 Ted Turner donated $1B to the United Nations for international peace and human dignity, calling it his best ever investment.

●

The founders of frontier AI companies have known about the obvious existential risks their AIs would pose humanity since before they invented them. Why did they not use their extraordinary talents and money to prevent these risks by building or improving scientific and ethics-based governing institutions before they started this "AI suicide race"? Why did they not give $1B to the United Nations for AI safety, like Ted Turner did in 1997 to improve human dignity?

●

The founders of frontier AI companies have known about the existential risks their AIs would pose to humanity since before they invented them. The "AI suicide race" that we're now in has been predicted since at least 1965. Why did AI leaders not use their time, talent, and money to prevent these risks by building or improving AI safety-focused global governance organizations before leading existentially risky research into frontier AI? For example, by donating to the United Nations for AI safety? For reference, in 1997 Ted Turner donated $1B to the United Nations for international peace and human dignity, calling it his best investment ever.

●

Why aren't AI leaders consenting to the 100K+ person call for a prohibition on artificial superintelligence until there is "broad scientific consensus that it will be done safely and controllably, and with strong public buy-in"? If they're claiming they worry about their competitors racing ahead, why don't they agree to conditionally pause their riskiest work if others do so as well? And then dedicate their time, talent, and money to develop global governance of AI first?

●

Why aren't AI leaders consenting to the 100K+ person call for a prohibition on artificial superintelligence until there is "broad scientific consensus that it will be done safely and controllably, and with strong public buy-in"? If they're worried about their competitors racing ahead, why don't they agree to conditionally pause their riskiest work if others do so as well? And then dedicate their time, talent, and money to develop global governance of AI first?

Call to Action

●

If you're able, will you consider mitigating existential or catastrophic risks from AI the number one daily priority for you going forward until we’ve collectively eliminated most of the risks? If no, why not? Is there anything that might change your mind? If you're extremely busy, can you commit to one small action per day? Many actions can be done in minutes with near zero financial cost. Getting started can be hard for some, but we’re happy to help. Once you consider yourself an existential safety advocate with a civic duty to preserve and protect humanity—as you might your family or nation—it may become easier to take action.

●

If not you, then who?