Nine countries now possess nuclear weapons. Arms control agreements painstakingly built over decades have collapsed. Geopolitical competition is intensifying rivalries between nuclear-armed powers. North Korea continues expanding its arsenal and delivery systems. China is on a trajectory to go from roughly 600 warheads today to around 1,500 by 2035. Meanwhile, Russia bears most of the responsibility for the expiration of New START and is developing an array of exotic new nuclear delivery systems designed to circumvent missile defenses. This includes nuclear-powered and nuclear-armed cruise missiles and underwater drones, hypersonic glide vehicles, and a nuclear-armed anti-satellite weapon. Although most recent focus has reasonably been on Iran’s nuclear program and the prospect of autonomous conventional weapons systems, the potential for the automation of nuclear launch decisions themselves in an age of artificial intelligence looms over the conversation.

One highly concerning possibility is that a country like Russia could automate the use of nuclear weapons. At first glance, this sounds like something that would only happen in a Hollywood movie, like the “doomsday machine” in Dr. Strangelove or “Skynet” in The Terminator. But this is not science fiction, and there is historical precedent for semi-autonomous nuclear launch systems.

During the Cold War, Moscow actually deployed a system called Dead Hand or Perimeter, which would monitor for a nuclear attack on Soviet territory and automatically delegate nuclear launch authority to a lower-level official operating the system if a communications link could not be made with Soviet high leadership. The purpose of Dead Hand was to guarantee that the Soviet Union would be able to retaliate against an American first-strike attempt with its own nuclear counterstrike, even if its leadership was killed. In other words, it was a fundamentally defensive system. Similarly, Adam Lowther and Curtis McGiffin have argued in War on the Rocks that America needs its own dead hand system to effectively deter nuclear attacks against Washington.

However, it is also plausible that automated nuclear systems could instead be used to pursue de facto offensive and revisionist objectives via compellence or deterrence. What if Russia invaded a country like Estonia or Poland and threatened to use nuclear weapons if NATO intervened? The goal of this kind of threat could be to deter NATO intervention to aid a country unjustly attacked or compel NATO members to back down from their Article 5 commitments. Ordinarily, such threats lack credibility due to the dynamics of mutually assured destruction and the nuclear non-use norm. Yet, if it could be implemented with computer-based automation, then the threat might become more believable.

A recent survey experiment we did of United Kingdom parliament members and the general public demonstrates that the risk of nuclear-armed states pursuing coercive ends by automating nuclear launch is worth taking seriously. Our study presented a scenario where Estonia was invaded by Russia, which then threatened NATO countries like the United Kongdom with a nuclear attack if they came to Tallinn’s aid. Alarmingly, when Moscow’s nuclear threats were implemented through automation, they were more credible and made respondents more willing to back down. Though incredibly dangerous, states may be tempted to create automated nuclear launch systems to more effectively coerce adversaries.

Why Nuclear Coercion Is Challenging

Coercion involves using threats against other states to accomplish political objectives without having to actually resort to costly uses of force. Since, as Robert Pape said, “nuclear weapons can almost always inflict more pain than any victim can withstand,” threatening nuclear punishment would ostensibly be an effective coercive strategy. But there are two reasons why it typically is not.

The first constraint is when mutually assured destruction exists between two nuclear-armed states. For threats to be credible, the target must believe there is a chance the punishment will be inflicted if their adversary’s red line is crossed. The problem is that if a target state has nuclear weapons and a secure second-strike capability, it could retaliate with its own devastating nuclear counterstrike. From a rational perspective, carrying out a nuclear threat would therefore be illogical and thus lack credibility under these circumstances.

The second constraint is the nuclear taboo or non-use norm, to the extent it exists. Given the stigma against using nuclear weapons — especially if there are other good alternatives — nuclear threats may lack credibility because states will expect the political, economic, and social costs of carrying out the threat to be too high.

How Automation Could Change the Game

Unless it is integrated into their programming, computers do not care if carrying out a nuclear threat is irrational because it could lead to nuclear retaliation or other significant costs. By developing and activating an automated nuclear weapons system, leaders can more credibly threaten to use nuclear weapons because delegating launch authority to a computer allows leaders to more literally tie their hands. As the infamous fictional character Dr. Strangelove said, “because of the automated and irrevocable decision-making process which rules out human meddling, the doomsday machine is terrifying … and completely credible and convincing.”

Of course, computer systems can malfunction and fail, meaning there would be substantial risk from automating nuclear command and control. During the Cold War, there were several near accidents where early warning systems falsely indicated a nuclear attack had been launched, which could have resulted in mistaken nuclear retaliation if a human had not detected the error. Modern computer systems are not immune to mistakes either. For instance, Boeing’s 737 MAX Maneuvering Characteristics Augmentation System was responsible for several fatal commercial airline crashes in 2018 and 2019, and automated trading systems contributed to the 2010 “flash crash,” where hundreds of billions of dollars of market value were wiped out in a matter of minutes.

Computer systems are also susceptible to hacking. Malign state or non-state actors could leverage this vulnerability to try and start a nuclear war between their enemies or erode a country’s second-strike capability. If states automate nuclear launch processes via complex and opaque machine learning programs based on artificial intelligence rather than hard-coded and rules-based systems like Dead Hand, then that might further increase the risks that humans will not understand how it is making decisions. Combined with automation bias — the tendency for humans to overly trust computers — this could lead to disaster.

Given these downsides, most leaders are unlikely to take the extreme step of automating nuclear use. Still, nuclear blackmail has been attempted historically by countries like the United States in the Taiwan Strait Crisis and Pakistan in the Kargil War. Automating nuclear launch may appear to be a rational strategy for several reasons.

First, a country that fears decapitation by a more powerful nuclear country may automate nuclear command and control to speed up its ability to retaliate and protect itself in case of attack. This is why Lowther and McGiffin believe the United States needs its own dead hand system. Second, the increasing speed of conventional war in an age of artificial intelligence could make countries fear rapid conventional defeat, making them more likely to want to automate nuclear command and control to preempt potential decapitation. Third, if a country is at a conventional disadvantage or faces deteriorating conditions on the conventional battlefield, it may turn to nuclear coercion and take the dangerous step of automating nuclear use to achieve its strategic objectives.

Nuclear Automation “Works” (To Some Extent)

To test the efficacy of automated nuclear threats in influencing the views of political leaders and the general public, we surveyed over 100 current U.K. Members of Parliament and over 1,000 British citizens. We presented subjects with a hypothetical scenario set in 2030 where Russia invaded Estonia and threatened to use at least one nuclear weapon against NATO if they intervened.

This scenario is disturbingly plausible. Russia has made nuclear threats against Ukraine and NATO, and at one point, U.S. intelligence agencies estimated that the likelihood of Russian nuclear use was as high as 50 percent if their lines in southern Ukraine collapsed. According to reporting in the New York Times, “One simulation [by the Biden administration] … involved a demand from Moscow that the West halt all military support for the Ukrainians: no more tanks, no more missiles, no more ammunition.” This is quite similar to the scenario we presented to respondents in our experiments.

We then randomly varied whether the nuclear threat from Russia was implemented with an automated nuclear launch system that would automatically attack if Russian sensors and early-warning systems detected a NATO intervention, or via normal human-controlled means. For all respondents, we then asked them whether they thought Russia’s threat was credible and whether they were willing to abide by Russia’s red line.

The results were sobering and show the way the age of AI could increase nuclear risk — automation made Russia’s threats more credible and effective. Members of the U.K. public were over 13 percentage points more likely to believe Russia would launch nuclear weapons if their red line was violated when the threat was implemented with automation than without. U.K. parliamentarians were also nine percentage points more likely to support backing down from protecting Estonia when Russia made an automated nuclear threat. It is striking that automated nuclear coercion could yield benefits even though the United Kingdom has its own nuclear weapons, and Estonia is a fellow member of NATO, guaranteed protection under Article 5.

While we specified the existence of an automated system to respondents in the scenario, the results raise the question of how a state would convince adversaries that they have automated nuclear launch, and the prospect of bluffing. Given the risks discussed above and the fact that political leaders are often loath to cede personal control over crucial policy decisions, governments may be perceived as unlikely to delegate the means of mass destruction to computers. Verifying the existence of an automated nuclear system would also be extremely challenging, as states are highly unlikely to share the source code of such programs for fear that their enemy might discover a weakness. The obstacle to making nuclear automation claims credible is further compounded by the fact that if nuclear automation provides coercive advantages, then states may have an incentive to bluff about the existence of such systems in a bid to gain the benefits without risking the costs.

However, given advances in frontier AI over the last few years — if not months — the notion that a state could automate nuclear launch could easily seem credible from a technical perspective. If the stakes are viewed as sufficiently high for the coercing country, such as if they are losing on the battlefield and leaders need to gamble for resurrection, then they may also be perceived as willing to attempt nuclear automation. Credibility may also be higher for autocratic regimes known not to trust their military. Public statements by leaders that automation systems exist and have been activated would also put their reputation on the line in a way that may make their claims seem credible. Over time, if leaders bluff and are caught in a lie, then future assertions about nuclear automation will likely be viewed with greater suspicion. Conversely, truthful automation threats today might enhance the credibility of similar threats tomorrow. Finally, states may gather intelligence from human sources or electronic intercepts that can help adjudicate whether statements about automation are accurate or not.

At the end of the day, claims about nuclear automation do not necessarily have to be believed with 100 percent certainty to be effective. Our survey results suggest that if states perceive that there is a reasonable non-zero probability that an adversary has taken the risky step of reducing human control over nuclear weapons, then that might make them sufficiently concerned to trigger greater consideration for backing down.

Several additional qualifications are in order with respect to our findings. First, the changes in attitudes we uncover (9–13 percentage points) are meaningful but modest. They show that nuclear automation can move the needle to some extent but also imply that it is far from a panacea for coercion. Large percentages of U.K. parliamentarians and members of the public do not change their views when presented with an automated threat.

Second, there was some inconsistency in the results. For example, although U.K. parliamentarians were more willing to back down when Russia made automated nuclear threats, they did not, on average, find automated threats more believable than non-automated ones. This indicates that the challenges of making automated threats credible (discussed above) are potentially real. Even if policymakers’ best guess about the probability of nuclear use was roughly the same for non-automated and automated nuclear threats, they may have been less confident and more uncertain about their estimates for the latter given the relative novelty of the technology and the inability to bargain with a computer in the same way one can bargain and reason with a human. Greater uncertainty about the risk of nuclear escalation may then have convinced some Members of Parliament to support backing down to avoid a devastating outcome.

Why Nuclear Automation Is Different

Given the coercive leverage automating nuclear launch systems can confer, states should prepare for the possibility that their adversaries might consider taking this perilous step. To deter the development and use of these systems in the first place, states could publicly declare they will impose costs — like economic sanctions — if they gather intelligence indicating that others are creating these kinds of systems.

For nuclear-armed states, bolstering their second-strike capabilities would be the surest way to avoid getting caught in an automated threat trap. Developing automated systems will be especially tempting for states that fear decapitation, as the Soviets did when they built Dead Hand. Nuclear modernization to reinforce second-strike capabilities could therefore reduce the incentives for states to pursue automation in the first place, at least for defensive purposes.

While arms control in the age of AI will be extremely hard since it is a general-purpose technology, one possible area of cooperation is a multilateral agreement prohibiting automated nuclear launch. Although compliance with this kind of agreement would likely be extremely challenging, if not impossible, to verify, a formal public commitment would increase the costs of noncompliance. While Washington and Beijing have informally agreed not to automate nuclear decision-making, a legally binding agreement with more nuclear-armed powers would provide stronger safeguards. Requiring that signatories also codify this commitment as part of their domestic law would further strengthen constraints against violation. Pursuing this kind of agreement could be a win for a Trump administration.

Still, despite these efforts, a revisionist country like Russia could attempt to leverage claims of nuclear automation to annex former members of the Soviet bloc. NATO countries should be engaging in wargaming at the strategic and operational levels to develop a playbook for how to respond to automated nuclear threats. The optimal response will also depend on how credible the threat is based on intelligence and structural factors, as well as the value placed on the territory being targeted.

Even if the coercive advantages of automated nuclear systems make their adoption tempting to some states, putting Armageddon on autopilot is a path NATO countries themselves should avoid. The stakes of preventing nuclear use are simply too great to accept the risks of malfunction, hacking, automation bias, and escalation that automated nuclear launch systems would bring.

There is much apprehension about the integration of automation and artificial intelligence into the military domain. The integration of AI into military systems is inevitable, especially in an era of precise mass where states are likely to invest in millions of aerial, ground, and sea drones that will be challenging to manage effectively without AI and autonomous systems. Integrating greater autonomy into drones — with appropriate safeguards — can reduce these systems’ susceptibility to electronic warfare while also avoiding the downsides of using wired systems. The destructive power of conventionally armed drones is also, of course, significantly less than that of nuclear weapons, making the risks of automated drones much smaller than those of automated nuclear launch systems. As Michael Depp and Paul Scharre wrote in these pages, some applications of automation can improve stability, whereas others can be dangerous and destabilizing.

When it comes to nuclear weapons, our research demonstrates that, unfortunately, there may be coercive benefits to automation, meaning the danger is real and concern is justified. But it is important to distinguish the level of risk for different types of military systems. Putting Armageddon on autopilot would entail large and unique risks compared to other AI-powered weapons systems.

Joshua A. Schwartz is an assistant professor of international relations and emerging technology at the Carnegie Mellon Institute for Strategy & Technology. He was previously a Grand Strategy, Security, and Statecraft fellow at the Harvard Kennedy School and the Massachusetts Institute of Technology.

Michael C. Horowitz is a senior fellow for technology and innovation at the Council on Foreign Relations and the Richard Perry Professor and director of the Perry World House at the University of Pennsylvania. He is the former U.S. deputy assistant secretary of defense for force development and emerging capabilities.

Image: Midjourney