Decisions in security contexts, including armed conflict, law enforcement, and disaster relief, often need to be taken under circumstances of limited information, stress, and time pressure. Since AI systems are capable of providing a certain amount of relief in such contexts, such systems will become increasingly important, be it as decision-support or decision-making systems. However, given that human life may be at stake in such situations, moral responsibility for such decisions should remain with humans. Hence the idea of “meaningful human (...) control” of intelligent systems. In this opinion paper, we outline generic configurations of control of AI and we present an alternative to human control of AI, namely the inverse idea of having AI control humans, and we discuss the normative consequences of this alternative. (shrink)

We propose a multi-step evaluation schema designed to help procurement agencies and others to examine the ethical dimensions of autonomous systems to be applied in the security sector, including autonomous weapons systems.

Predictability is a prerequisite for effective human control of artificial intelligence (AI). The inability to predict malfunctioning of AI, for example, impedes timely human intervention. In this paper, we empirically investigate how AI’s predictability compares to the predictability of humans in a real-effort task. We show that humans are worse at predicting AI performance than at predicting human performance. Importantly, participants are not aware of the differences in relative predictability of AI and overestimate their prediction skills. These results raise doubts (...) about the human ability to effectively exercise control of AI — at least in certain contexts. (shrink)