Trustworthiness in Autonomous Weapon Systems: A Military Doctrine Imperative

Introduction

Autonomous Weapon Systems (AWS) represent a transformative leap in military technology, enabling advanced capabilities in surveillance, target acquisition, and engagement. These systems, ranging from loitering munitions to cruise missiles, are integral to modern warfare. However, their deployment in high-stakes environments demands strict adherence to ethical and operational standards to ensure effectiveness and moral acceptability. Central to this is the concept of trustworthiness, which encompasses technical robustness, safety, data governance, transparency, and human oversight. Many law and technology researchers argue that AWS must be regulated to ensure trustworthiness, emphasizing the need for legal frameworks to enforce ethical standards and accountability. This content argues that AWS are already developed with trustworthiness as a priority because military doctrines inherently require robustness and human-centricity to execute successful operations. By examining key military policies and specific AWS examples, I demonstrate how these systems are engineered to align with doctrinal mandates, ensuring reliable performance and human control, while also addressing the regulatory concerns raised by researchers.

Photo by Sergey Koznov on UnsplashMilitary Doctrines: Mandating Robustness and Human-Centricity

Military doctrines worldwide emphasize two critical characteristics for AWS: robustness and human-centricity. Robustness refers to a system’s ability to perform reliably under diverse and challenging conditions, such as electronic warfare, cyber-attacks, or unpredictable environments. Human-centricity ensures that human operators retain oversight and control over critical decisions, particularly those involving lethal force. These characteristics are not optional but are codified in military policies to guarantee operational success and ethical compliance.

A prominent example is the United States Department of Defense (DoD) Directive 3000.09, titled “Autonomy in Weapon Systems,” effective January 25, 2023. This directive mandates that autonomous and semi-autonomous weapon systems be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force. It further requires rigorous verification, validation, and testing (V&V and T&E) to ensure system reliability in realistic operational conditions against adaptive adversaries. The directive also emphasizes system safety, anti-tamper measures, and cybersecurity, aligning with the robustness requirement. Additionally, it calls for understandable human-machine interfaces and adherence to DoD AI Ethical Principles, which include being responsible, equitable, traceable, reliable, and governable. These stipulations underscore that trustworthiness — encompassing reliability, safety, and human oversight — is a doctrinal necessity.

While the DoD directive is U.S.-specific, similar principles likely guide other advanced militaries, as evidenced by the design of AWS developed by countries like Israel and Turkey. The prioritization of robustness and human-centricity in these doctrines ensures that AWS can operate effectively while maintaining accountability, thereby making trustworthiness a core attribute of their development.

Defining Trustworthiness in AWS

Trustworthiness in the context of AWS is a multifaceted attribute that includes:

• Technical Robustness: The ability to function reliably in adverse conditions, such as electronic countermeasures or cyber-attacks.
• Safety: Mechanisms to prevent unintended harm, such as fail-safes or self-destruction protocols.
• Data Governance: Secure handling of sensitive operational data through encryption and strict protocols.
• Transparency: Providing operators with real-time feedback and explainable AI to understand system decisions.
• Human Oversight: Ensuring human operators can intervene and control critical decisions, often through human-in-the-loop (HITL), human-on-the-loop (HOTL) or human-in-command (HIC) mechanisms.

These elements collectively ensure that AWS meet the operational and ethical demands of military doctrines. By embedding these features, developers address the doctrinal requirements for robustness and human-centricity, making trustworthiness a foundational aspect of AWS design.

Case Studies: AWS Embodying Trustworthiness

Several AWS currently in use or under development exemplify how trustworthiness is prioritized to meet military doctrinal requirements. Below, we analyze four systems: the IAI Harop, STM Kargu 2, Roketsan Çakır, and Anduril Roadrunner. Each system demonstrates features that align with robustness and human-centricity, supporting the argument that trustworthiness is a priority driven by military needs.

IAI Harop

The IAI Harop, a loitering munition developed by Israel Aerospace Industries, is designed to autonomously identify and engage targets in high-risk environments, particularly for Suppression of Enemy Air Defenses (SEAD) missions (IAI Harop). Its robustness is ensured through advanced algorithms that enable operation in areas subjected to electronic countermeasures or cyber-attacks. The system’s electro-optical (EO) sensors and man-in-the-loop control mode allow it to loiter for over six hours, conducting intelligence, surveillance, and reconnaissance (ISR) before engaging targets. Safety features include self-destruction mechanisms to prevent unintended damage in case of system failure or loss of control. Data governance is maintained through secure encryption and military protocols, ensuring the integrity of operational data.

Transparency is achieved through real-time operational feedback, enabling thorough oversight and post-mission analysis. Crucially, the Harop requires human validation of targets, with operators selecting static or moving targets detected by the EO sensor. This human-centric approach ensures compliance with engagement rules and minimizes collateral damage, aligning with doctrinal mandates for human oversight. The Harop’s design reflects a strategic integration of autonomy and human control, prioritizing trustworthiness to meet military operational needs.

STM Kargu 2

The STM Kargu 2, a rotary-wing kamikaze drone developed by the Turkish defense company STM, is tailored for asymmetric warfare and counterterrorism operations. Its robustness is evident in its advanced target recognition algorithms, which enable reliable performance in electronic warfare scenarios. Fail-safe mechanisms enhance safety and mission integrity, preventing unintended engagements. The Kargu 2 employs secure encryption and robust communication protocols to manage sensitive operational data, adhering to strict military and legal standards.

Transparency is a key feature, with the system providing detailed feedback and explainable AI behavior, allowing operators to gain insights into its decision-making processes. This transparency fosters trust and supports informed decision-making. The Kargu 2 is human-centric, enabling operators to maintain control over mission-critical decisions, ensuring compliance with ethical standards. By incorporating these features, the Kargu 2 demonstrates that trustworthiness is prioritized to meet the doctrinal requirements of robustness and human oversight, ensuring effective and ethical operations.

Roketsan Çakır

The Roketsan Çakır, a next-generation cruise missile developed by Turkey, is equipped with advanced autonomous guidance systems that adapt to dynamic operational environments. Its robustness is ensured through sophisticated navigation tools, such as GPS and terrain referencing, which allow it to function in contested conditions. Data security is paramount, with advanced encryption and strict governance policies protecting mission-critical information. While specific AI operations remain classified, the Çakır provides extensive mission feedback and elements of explainable AI to enhance transparency, fostering trust among military operators.

Human oversight is integral, with the Çakır operating under HITL, HOTL, and HIC governance mechanisms. These allow operators to control mission parameters and intervene when necessary, ensuring adherence to ethical standards and rules of engagement. The Çakır’s design reflects a balance between advanced autonomy and human control, prioritizing trustworthiness to meet military doctrinal requirements for reliable and accountable operations.

Anduril Roadrunner

The Anduril Roadrunner, an autonomous reconnaissance and strike system developed by Anduril Industries, excels in real-time threat assessment and decision-making in complex environments. Its robustness is achieved through adaptive algorithms that adjust to evolving threats, complemented by safety protocols like self-destruction to minimize risks of collateral damage. Secure data handling is ensured through encryption and strict governance protocols, maintaining the confidentiality and integrity of mission-critical information.

Transparency is maintained via real-time telemetry and explainable AI, providing operators with a deep understanding of operational decisions and system performance. The Roadrunner operates under HITL, HOTL, and HIC governance mechanisms, empowering operators to oversee and control key decisions. This human-centric approach ensures ethical operations and compliance with military standards, demonstrating that trustworthiness is a priority driven by doctrinal mandates for robustness and human oversight.

Alignment with Military Doctrines

The features of these AWS align directly with the requirements outlined in military doctrines, such as DOD Directive 3000.09. The table below summarizes how each system meets the key doctrinal requirements of robustness and human-centricity, reinforcing the argument that trustworthiness is prioritized in their development.

Table 1: The Alignment of AWS with Requirements Outlined in Military Doctrines

Table 1 illustrates that each system incorporates features that ensure reliable performance and human control, directly addressing the doctrinal mandates for robustness and human-centricity. These features collectively contribute to the trustworthiness of the systems, enabling them to meet the operational and ethical demands of modern militaries.

Ethical and Regulatory Context

While AWS are often exempt from civilian regulatory frameworks, such as those developed by the European Union, military organizations impose their own stringent requirements to ensure trustworthiness. The exemption from civilian regulations is based on the consensus that such oversight could hinder development and potentially compromise national security. However, as evidenced by DOD Directive 3000.09 and the design of systems like the Harop and Kargu 2, militaries prioritize trustworthiness through internal policies that mandate robustness, safety, and human oversight. This self-regulation ensures that AWS operate effectively under all conditions while addressing ethical concerns, such as minimizing collateral damage and maintaining accountability.

The ethical debate surrounding AWS remains complex, with concerns about the loss of human judgment in lethal decisions. However, the incorporation of HITL mechanisms in systems like the Harop and Çakır suggests that developers are addressing these concerns by prioritizing human-centricity, as required by military doctrines. This balance between autonomy and human control is a key factor in building trust among stakeholders, including operators, policymakers, and the public.

Conclusion

The development of Autonomous Weapon Systems with a focus on trustworthiness is a direct result of military doctrines that demand robustness and human-centricity for operational success. Policies like DOD Directive 3000.09 mandate that AWS be reliable, safe, and subject to human oversight, ensuring they can perform effectively in challenging environments while maintaining ethical standards. Systems such as the IAI Harop, STM Kargu 2, Roketsan Çakır, and Anduril Roadrunner exemplify this prioritization through their robust algorithms, safety mechanisms, secure data handling, transparency features, and human-centric governance. By embedding these elements, developers meet the rigorous standards set forth by modern militaries, ensuring that AWS are trustworthy tools for effective and ethical military operations. As AWS continue to evolve, the integration of trustworthiness will remain a cornerstone, driven by the enduring requirements of military doctrines.

Trustworthiness in Autonomous Weapon Systems: A Military Doctrine Imperative was originally published in Coinmonks on Medium, where people are continuing the conversation by highlighting and responding to this story.