The concept of “meaningful human control” has been repeatedly raised during this week’s discussions. Many countries have said they support meaningful human control, while others have said they would like to explore the idea further. Still others have said that the concept of “meaningful human control” is vague and unhelpful.
As part of our ongoing research on autonomous weapons at the Center for a New American Security, we have examined the concept of meaningful human control, and while we did not originate this concept, we would like to offer some ideas for consideration, particularly with regard to how control is exercised in weapons today.
If the international community is concerned about a future in which autonomy might cause us to lose human control – whether the nature of that control is considered to be meaningful, effective, adequate, etc. – we thought it would be useful to understand what might be different between the weapons of today and future, lethal autonomous weapons.
We find that meaningful human control in weapons today has three essential components:
First, human operators make informed, conscious decisions about the use of weapons.
Second, human operators have sufficient information to ensure the lawfulness of a particular action, given what they know about the target, the weapon, and the action’s context.
Third, the weapon is designed and tested in a realistic operating environment, and human operators are properly trained to ensure effective control over the use of the weapon.
These standards of meaningful human control help to ensure that human operators and commanders are making conscious decisions about the use of force, and that they have enough information when making those decisions to remain both legally and morally accountable for their actions. Furthermore, appropriate design and testing of a weapon system, along with proper training for human operators, helps to ensure that weapons will be controllable and that they will not pose unacceptable risks.
These are important concepts for understanding how human control is currently exercised in weapons that are not autonomous. However, it is not to say whether the concept of meaningful human control is useful or necessary, or whether the concept should ultimately be adopted by the CCW. Throughout the course of this week, states will need to consider the appropriate framework for thinking about human control over autonomous weapon systems.
We hope that these points are useful for states as they continue to think about the nature of human control over weapon systems. More information about CNAS research on this topic can be found in our reports on autonomous weapons and meaningful human control, which are available on our website at http://www.cnas.org/ethicalautonomy.
More from CNAS
CommentaryThe Militarization of Artificial Intelligence
Militaries are racing to adopt artificial intelligence (AI) with the aim of gaining military advantage over competitors. And yet, there is little understanding of AI’s long-te...
By Paul Scharre
CommentaryAre AI-Powered Killer Robots Inevitable?
In war, speed kills. The soldier who is a split second quicker on the draw may walk away from a firefight unscathed; the ship that sinks an enemy vessel first may spare i...
By Paul Scharre
CommentaryThe Iranian Missile Strike Did Far More Damage Than Trump Admits
Over 100 American soldiers have been treated for traumatic brain injuries following Iran’s missile strike on Al Asad Air Base in western Iraq. The strike came in retaliation f...
By Loren DeJonge Schulman & Paul Scharre
PodcastEpisode 26 - Paul Scharre
What are autonomous weapons systems? How are they used in modern warfare? And how do we strengthen international cooperation? In this episode, the Director of the Technology a...
By Paul Scharre