Center for a New American Security

Human Rights Watch recently put out a report demanding a ban on fully autonomous weapons system and more scrutiny, as well as additional legal controls, to regulate the development and proliferation of robotic weapons. Human Rights Watch wants an international treaty prohibiting weapons that either “deliver force under the oversight of a human operator who can override the robot’s actions,” and “robots that are capable of selecting targets and delivering force without any human input or interaction.” In their report, seeking to advance policy to prevent indiscriminate warfare, they instead perpetuate a large degree of misperceptions about the way in which militaries operate in addition to needless fear-mongering about fully autonomous weapons which are highly unlikely to ever exist.

The first major problem is that HRW even has a category of “human-out-of-the-loop weapons” which are supposedly going to enter modern warfare. This is, needless to say, a logically ludicrous concept. No weapon is fully capable of taking humans out of the loop, unless it is part of its own command structure. The distinction HRW draws between “Human-on-the-Loop Weapons” and “Human-out-of-the-loop” weapons is totally arbitrary, particularly since one key criteria HRW draws is that “human-on-the-loop” weapons have human oversight which can override and veto their actions, and “human-out-of-the-loop” weapons do not.

Unless the U.S. were to design a weapon that, upon activation, simply began doing whatever its programming told it to do and nothing else, a “human-out-of-the-loop” weapon does not and will not exist. Furthermore, there is absolutely no reason for a human to want to deploy such a weapon. Correct me if I’m wrong, but what commanding officer wants to swap out his subordinates with a machine that is going to be less responsive to his orders?

Weapons fall into a command structure. Every weapon, regardless of its level of autonomy, will conduct missions designed by humans and carried out under human orders, supervised by humans with superior power over it. Indeed, comparable to human subordinates, a “human-on-the-loop” weapon gives a commander more opportunities to micromanage combat performance. If anything, a commander has fewer opportunities to scapegoat subordinates for the actions of an autonomous system.

HRW worries victims of “fully autonomous” weapons cannot confront those who have wronged them in court, which somehow obviates accountability through commanding officer. Supposedly, the entity pulling the trigger is essential to the aversion and prosecution of war crimes. But in this sense, robots do not change much of anything. Artillery gunners and their commanding officers, for example, frequently lack the information necessary to assess whether their fire mission is fully lawful or ethical. They are dependent on the wisdom of the people calling in and ordering the strike. The pilot of an F-16 flying at hundreds of miles an hour frequently lacks adequate ability to judge whether his target, particularly infrastructure targets, are legitimate or not. He relies, as a robotic aircraft would rely, on the wisdom of those who collected the intelligence on his targets, who have eyes on it from the ground who, if necessary, can correct how he deploys his munitions.

There is nothing inherently indiscriminate about an autonomous weapon, even if we assume it is going to face permanent inability to assess every single criterion of discriminate force vis-a-vis a human infantryman. An autonomous weapon using conventional munitions ought be assessed contextually. A weapon or munition that is discriminate for destroying a tank battalion in the open is probably not discriminate for clearing snipers out of a populated urban center.

Some weapons are so indiscriminate in a range of normal military contexts, and indeed without redeeming virtues of strategic efficacy that might justify them as proportional instruments, that banning them is relatively effective and prudent. There is very little discrimination possible with a chemical weapon whose physical nature makes selecting individual targets nigh impossible, or a biological weapon, which, once deployed, will continue operating fully autonomously with no possible human input. Not only that, but these weapons were so frequently operationally or strategically useless - and indeed, very dangerous to one’s own side - that it was entirely reasonable to put in place an outright prohibition. Even then, many militaries frequently commit violations of chemical weapons protocols by employing less-lethal gasses such as C-series agents and white phosphorous that have legitimate uses when it seems tactically prudent.

The attempt to blanket ban autonomous weapons relies on a blanket presumption of failure to discriminate that fails to take into account the way militaries operate. A commanding officer deploying autonomous weapons should know the limits of his system. An unmanned aerial system which can evade and engage hostile targets should not be allowed to select target types such as civilian vehicles or groups of individuals, nor should an autonomous weapon which cannot distinguish between civilians and soldiers with high enough reliability be emplaced in a city. This is just as we would not permit a jet attack aircraft to select and engage its own ground targets in a similarly populated area.

Autonomous weapons receive orders and can be programmed with rules of engagement. If these safeguards fail occasionally, this is not a particularly convincing argument. After all, look at the record of U.S. attempts to enforce roadblocks in Iraq. An infantryman may, seeing a civilian vehicle speeding towards his checkpoint, kill civilians in error because he is tried or concerned with protecting his life and those of his fellow soldiers. Humans disobey orders and make judgment calls about ROE or commander’s intent all the time, whether they are in or out of the loop of their CO. Indeed, we could select a great number of alternate scenarios where a robot that has no fear for its own life, and no programmed ability to refuse or deviate from orders, may be more willing to enforce a strict ROE to the letter. This is not to impugn human combatants or to praise robots, but to note that autonomous weapons, like all weapons, will have limits and advantages.

So what if a commander cannot discipline or punish a robot? He can do things that he cannot do to a human deviating from orders. He can override its actions. Even if that mechanism fails, he could remotely self-destruct or destroy the robot in the midst of its commission of war crimes. A robot Calley is, in many ways, easier to deal with than a human war criminal. It is much less ethically difficult to deactivate or destroy a malfunctioning robot than to kill one of your men or women. Not only that, but the information collected and stored by a robotic weapon would prove much more useful in the prosecution of a war crime committed by a robot-operating unit than the testimony of soldiers who must grapple with the limits of human sense, psyche, and loyalty to each other.

HRW’s argument, then, seems so overbroad as to likely be utterly ineffectual. Much as when Britain tried to ban submarines during the Washington Naval Treaty as being inherently indiscriminate and criminal because of the specific role they played in World War I, no power with the capability to take advantage of the huge military benefits of adopting these weapons is likely to forgo them for a blanket treaty, if they even buy into such a treaty at all.

One might justify HRW’s piece as starting a conversation - and I join that conversation by saying the specifics of their proposal and the view they adopt of autonomous technology are utterly ill-considered. There can and should be limitations on the way weapons can be used. But for the great majority of weapons in the human arsenal, these need to be thought of contextually rather than rigidly. Like it or not, autonomous weapons are already present, and states are going to use weapons with considerably efficacious attributes. But measuring the legality of autonomous weapons against higly specific scenarios against the standards that very often seem to ignore how militaries and the human beings in them behave on the battlefield is the wrong way to start this debate, and certainly not a sound foundation for credible regulations of their use. There are many reasons to start a debate. What HRW appears to aim to do is to strangle it in the crib on the basis of hyperbolic supposition. I strain to think of arms control beginning from such a premise which has had lasting or beneficial effect.

Reading Dan's latest drone piece, I was reminded of some arguments about airpower and information warfare I first encountered in 1999 while watching the Kosovo War on TV.  About 15 years ago, you couldn't pick up a defense or international relations publication without reading a downbeat analysis of "virtual" or "post-heroic" war. Modern technology had made wars cost-free, more likely, and could potentially subvert democracy, the theme went. Today we're hearing a similar jingle, and this time, the future of robotics technology is the bogeyman.

The 1990s literature on virtual war and post-heroic war was not uniformly bad. Some of these reflections were thoughtful, as Jack McDonald notes about Michael Ignatieff's always fascinating oeuvre. Others were completely ridiculous and sprung from intellectuals' loathing and suspicion of any and all standoff weaponry. Perhaps the nadir of this era were Jean Baudrillard's arguments about the Gulf War. Yes, most people got what it really meant wrong. But even when you take into account the points Baudrillard really wanted to make about the relationship between media, information-based warfare, and politics, he was still a tad off course. The post-heroic moment was fundamentally rooted in a deeply rooted sense of fear about the implications of the information-based Revolution in Military Affairs (RMA). It seemed to cement the dominance of the West and the US in particular. Worse yet, it promised a future of war in which Third World armies were swatted away by precision weapons as surely as a little boy could fry a helpless nest of ants with a magnifying glass. 

The post-heroic critics were wrong on multiple levels. First, they were wrong (as they are now) about the novelty of power projection enabling discretionary wars. As Dan has noted, if you're unhappy about discretionary wars, blame the Navy. If you're unhappy about privatized force being used as tools of state power, well, blame John Hawkood. David Parrot has also written about how privatized force has been the norm for most of Western history. Private CIA air forces supporting local allies is the Agency's equivalent of a cheap bar band playing "Free Bird," as is the peculiar idea held by many supporters of drone operations and humanitarian intervention that operating in an a country's airspace without its expressed permission is not violating its sovereignty. It's true that America's intervention in Libya had troubling implications for war powers---but I'm talking about the first one 200 years ago. You know, the one fought by a crew of foreign mercenaries Uncle Sam dredged up from every sundry Mediterranean watering hole from Athens to the Levant.

Second, the "virtual war" movement took Pentagon ad copy seriously without realizing that all of the "systems of systems" rhetoric was an aspiration rather than a military fact. Most postwar analyses of the actual impact of RMA platforms during the Gulf War also cast doubt on the idea that platform superiority would translate into operational effectiveness. The grisly ambush of Blackhawk Down also proved that special operations forces aren't always so special when they are outnumbered by massess of Third World infantry in urban environments. Finally, the Future Combat Systems (FCS) fiasco and the numerous tactical surprises encountered by ground forces in the 2003 invasion phase of the Iraq War laid the final nail in the coffin in the technocentric vision of dominant battlespace knowledge that would supposedly "lock in" American military advantage. This didn't mean that an RMA hadn't occured, or that Admiral Cebrowski and others were not correct about the way technology was transforming military operations. But those that feared the political impact of these technologies, at home and abroad, had made future projections that did not take into account the zig-zaggedy trajectory of strategic history.

Military revolutions also do not remain within a single country. The Chinese were so shocked by the seemingly effortless way the US intervened in the Middle East and Balkans during the 1990s and resolved to invest in "informatizing" their forces and building up an anti-access capability. People's Liberation Army (PLA) military writings, though shrouded in the political jargon characteristic of Communist militaries, could have been written by a DoD green-badger during the height of the Effects-Based Operations (EBO) craze. Yes, PLA doctrine, is, like a particularly inspired Houhai Bar Street cover band, partially inspired by the top 40 "hits" of yesteryear.  But the specific way the PLA built its own information warfare, precision-strike, and joint operations doctrine bears little resemblance to the American assimilation of the RMA. Quite naturally, it's shaped by unique Chinese roles and missions, technological base, doctrine, and strategic culture. And this is to say nothing of the way technology may have transformed non-state militaries.

Military innovation literature has consistently demonstrated the basic fact that individual national culture, economics, and defense requirements dictate the form a particular state's use of technology or doctrine will take. How a state or non-state actor assimilates a given technology matters more than whether they have it or not, especially since technological innovations do not necessarily stay with first adopters. Examples can be found in the evolution of tank doctrine and carrier operations. The British may have been initially ahead in tank design and doctrine, and but that didn't save them from deploying them in a manner distinctly inferior to the Wehrmarcht. The fragility of RMAs was grasped by the early RMA writers, who often warned that the US could not expect to be the only actor to exploit precision-strike technology.

Another basic truth is that those who forecast continued impunity also forget that the defense is the stronger form of war. Tanks soon were countered by layered anti-tank defenses. Massed artillery barrages in World War I were frustrated by elastic defense. Aircraft once prophesized to be invincible were torn to pieces over the skies in Europe in the Combined Bomber Offensive. And what of the nuclear weapon, the so-called "ultimate weapon?" Well, its ultimate-ness meant that it could not be employed as an effective operational weapon, and using nuclear weapons to do anything more than deter was very chancy. For every ambiguous success of nuclear compellence (such as the end of Pacific War and the Korean War), there have also been many more failures to achieve escalation dominance. Iran is likely to discover that all they do is protect it from something the US has never seriously contemplated: full-bore regime change. And covert action, the political "ultimate weapon" of the Cold War era, has a checkered history so infamous that even reasoned scholars of intelligence history such as John Prados think the entire idea in and of itself is dubious. Maybe Prados is a bit too harsh, but his point is important.

Solly Zuckerman noted in 1962 that technological complexity actually make it more difficult to achieve strategic effect on the battlefield. More complex systems tend to increase personnel requirements and require a complex backbone of supporting technologies and systems to optimize. This will not change with the introduction of autonomous weapons. In fact, the need to ensure that these military technologies act in accordance with the commander's intent will likely create new classes of technicians and operators--just as remotely piloted vehicles generated their own set of support requirements. Zuckerman also argues that more complex technologies also limit freedom of action, which is also true when one examines the political implications of actually utilizing drones in expeditionary environments. Autonomous weapons, should they emerge, will be targeted by a group of international lawyers and activists opposed to their mere existence. Using an autonomous system is one thing on the open seas or skies, but even the most gung-ho robotics supporter is likely to see their relative lack of utility in a populated area. As Martin Libicki predicted in The Mesh and the Net, urban warfare will remain an infantryman's game. Sure, technology will transform infantry operations, but all of the robotic bells and whistles will not change the fact that the ultimate decider of the "three-block war" is the man on the scene with the gun.

Maybe future robotics technology will expand impunity, but strategic history suggests that impunity never lasts very long. The predictions of "virtual war" during the 1990s must have seemed very hollow to Army and Marine soldiers slugging it out house-to-house in Fallujah or armor commanders discovering in 2003 that they were still fighting encounter battles in the age of dominant battlespace knowledge. Critics of American intervention and the Shock and Awe crowd are strangely both (unintentionally) in agreement about the future of warfare. But the future, as that great philosopher James T. Kirk once said, is an undiscovered country.