Machine-to-Machine Deception and Perfidy: Which Signal, What Protection?

Machine-to-machine deception
Getty images for unsplash.com

Estimated reading time: 12 minutes


Article 37 (1) of Additional Protocol I (AP I) prohibits killing, injuring or capturing an adversary by resort to perfidy. Perfidy means acts “inviting the confidence of an adversary” that protection under international humanitarian law (IHL) applies, “with intent to betray that confidence”. The US Department of Defense (DoD) does not accept that customary law bars capture by perfidy (DoD Law of War Manual, § 5.22.2.1, at p. 329). My companion post on deepfakes and perfidy separated two layers in machine-generated deception aimed at people. The first is a borrowed identity. The second, present only in some cases, is a claimed protection. Article 37 reads the protection a deception claims, not the identity it borrows. Such deception is therefore perfidious only when its content feigns protection under IHL. This post asks whether the same test governs machine-to-machine deception.

Consider an air-defence network that interrogates an approaching aircraft. The transponder answers with a code reserved for medical aircraft, and the network does not engage. Minutes later, the aircraft attacks. Has its crew committed perfidy, and against whom? This post addresses deception aimed at an adversary’s machines; deception aimed at people was the subject of my prior companion post. This post also covers deception that reaches a human through a machine, where the system’s processing gives the deception its effect.

Perfidy and ruses are governed by customary rules in international and non-international armed conflicts alike. The International Committee of the Red Cross (ICRC) study of customary IHL confirms this (Rules 65 and 57). Distinctive signals, discussed in Section III, are different. They are an AP I instrument, and the customary rule speaks only of the distinctive emblems (Rule 59). For signals, the analysis is therefore confined to international armed conflict.

I. Two Questions, Not One

Article 37 does not prohibit deception. It prohibits one kind of deception: betraying an adversary’s confidence in the protection that IHL affords. Article 37 (1) AP I defines perfidy as “[a]cts inviting the confidence of an adversary to lead him to believe that he is entitled to, or is obliged to accord, protection under the rules of international law applicable in armed conflict, with intent to betray that confidence”. Its four examples begin with “the feigning of an intent to negotiate under a flag of truce or of a surrender”.

The Addressee Debate

The fullest case for a functional reading of Article 37 comes from Samuel White, Jonathan Kwik and Dora Velenczei. I set it out here and engage it below. Read purposively, they argue, the provision covers deliberate, bad-faith deception of an adversary’s AI system. “The law does not require a human to be deceived,” they conclude (at “Conclusion”). The test becomes whether the target system is induced to act consistently with the obligation to respect protection. A system that classifies an object as protected and holds back an otherwise lawful attack has acted on the signal. The authors thus recast the invitation of confidence as an invitation of compliance (at “A Move Towards Functionality”). They set emergent behaviour aside and leave the line between electronic warfare and perfidy open (at “Conclusion”).

The addressee question is older than their article. In 2017, the experts behind the Tallinn Manual 2.0 (hereinafter “Tallinn 2.0”) divided on it (Rule 122, para. 9). The majority held that the confidence of a cyber system can be betrayed. The others thought confidence requires human involvement. Sean Price defends that minority view. For him, both “adversary” and “confidence” imply a human being (at Part III).

The Question the Debate Skips

In my view, the debate runs together two questions. One is whether a machine can be the addressee of perfidy. The other is whether the deception uses a protective signal at all. Sorting cases by the second question shows how rarely the first decides anything. A deception that borrows the adversary’s own identifiers claims no protection, so the addressee does not matter. A deception that uses a protective signal falls under Article 38, whatever its addressee. If a human reads the result and killing, injury or capture follows, Article 37 applies as well. The addressee question decides only a residual. It consists of machine-only chains with no protective signal and no human perceiving a claim of protection.

The sorting is not new in form. In 2001, Gregory O’Brien proposed a two-step test for information operations (at “Impact on IO-based deception operations”). He wrote as a US Navy judge advocate in a personal capacity. His first step asks whether a protective sign, signal or symbol is used. The second asks whether a neutral State or an enemy is simulated. This post adds the addressee question and shows where it decides the case.

II. Enemy Identifiers: No Protection Claimed

Spoofing an adversary’s own identifiers is a ruse, whether a person or a system receives it. Article 37 (2) permits ruses because they do not invite confidence with respect to protection. The DoD manual applies this directly. It allows enemy codes, passwords and countersigns to be used in a ruse (§ 5.23.1.5, at p. 332). The ban on enemy insignia, it explains, covers only concrete visual objects. Other treatments agree. The ICRC Commentary counts enemy wavelengths and codes used for false instructions among ruses (Article 37, para. 1521, at p. 443). Tallinn 2.0 lists false computer identifiers and enemy codes, signals and passwords (Rule 123, para. 2). O’Brien reaches the same result for enemy electronic emissions, on the same visual-objects reasoning (same section of his paper).

In operational terms, the reason is structural. An enemy identifier claims no protection under IHL. It borrows an identity the adversary is expected to guard. Nothing in the deception invites confidence in protection, so the addressee question never arises. This is the borrowed-identity layer of my prior companion post. That post found that the law has long treated this layer as a ruse. The only change is that a system, rather than a person, now processes the identifier.

Neutral identifiers are different. Article 39 (1) AP I prohibits using the “flags or military emblems, insignia or uniforms” of neutral States. Whether that reaches electronic identifiers is unsettled. The ICRC Commentary treats emblems of nationality as visible signs (Article 39, para. 1578, at p. 468). O’Brien thought simulating a neutral’s signals improper and possibly unlawful (same section of his paper). Article 39 raises its own questions, so the ruse finding above covers enemy identifiers only.

III. Protective Signals: Article 38 Before Article 37

Article 38 first

Where a deception uses a protective signal, Article 38 decides the case before the addressee question arises. Article 38 (1) AP I prohibits improper use of the distinctive emblem and of “other emblems, signs or signals provided for by the Conventions or by this Protocol”. Unlike Article 37, this prohibition is absolute. Any improper use is barred, whether or not it kills, injures or captures (ICRC Commentary, Article 38, para. 1532, at p. 448). Tallinn 2.0 takes the same view of its rule on protective indicators (Rule 124, para. 3). Jonathan Kwik and Adriaan Wiese draw the consequence for machines. Under the misuse rule, they argue, it does not matter whether the deceit reaches a human or an AI system. The misuse is itself the violation (at “Old Rules, Novel Legal Questions”). Price insists on human confidence for perfidy. Yet he, too, accepts that the rules on improper use of emblems apply to deception in cyberspace (at Part III.B). On this point, both sides of the addressee debate converge.

Signals Built for Equipment

Some of the treaty’s protective signals are read by equipment, not by the eye. AP I defines a distinctive signal as a signal or message specified in Annex I for identifying medical units or transports exclusively (Article 8(m)). The rules against misuse of the emblem apply to these signals as well (Article 18(8)). Annex I provides for identifying medical aircraft by secondary surveillance radar (SSR). A mode and code are reserved for their exclusive use (Article 9 (1); Article 8 before the 1993 amendment). Even the emblem may be made of materials that technical means of detection, such as infrared instruments, can recognise (Article 5 (3)).

These signals exist because weapons outranged the eye. Writing in 1982, the ICRC technical adviser Philippe Eberlin explained that missiles with homing devices could engage beyond the visual range of the emblem (at p. 202). SSR transponders reply automatically to interrogation, and the aircraft appears on the controllers’ radar screens (at p. 213). As I see it, this settles one part of the debate. The treaty has long carried protection through signals that equipment decodes. A machine link in the chain therefore does not by itself break the invitation of confidence. Whether a human must stand at the end of that chain is a separate question, and these provisions do not answer it.

Article 37 Where a Human Reads the Screen

Where the false signal reaches a human, Article 37 applies without the functional theory. O’Brien gave the paradigm case in 2001. In his view, transmitting a medical aircraft’s identification code to open a corridor through enemy air defences would be unlawful (same section of his paper). He does not say which rule it breaks. On the analysis here, Article 38 is breached by the misuse alone. Article 37 is engaged once an operator reads the false code and killing, injury or capture follows. White, Kwik and Velenczei accept that a pilot who broadcasts protected signals to deceive an air-defence system already falls within the prohibition (at “Novel Features of Machine-Mediated Deception”). In the introductory case of this post, the misuse of the medical code therefore breaches Article 38 in any event. If an operator saw the code, the attack is also perfidy on either view of the addressee question. If the network held fire with no operator involved, the case falls into the residual discussed next.

A digital emblem would fit the same pattern. The ICRC is developing one for medical and humanitarian digital assets. It intends to work with States on integrating it into IHL once the technical standards are final (Samit D’Cunha and Mauro Vignati, at “Anchored in standards”). Until then, how its misuse fits within Article 38 remains open.

IV. The Residual

Where the Addressee Question Decides

The addressee question decides a case only where no protective signal is used and no human perceives a claim of protection. White, Kwik and Velenczei describe the paradigm. An attacker crafts inputs so that a classifier treats a target as protected, without any recognisable signal (at “Novel Features of Machine-Mediated Deception”). On their functional reading, the mischief Article 37 targets is present here too, and the intent to betray decides the case. The false emblem serves them only as an analogy for the effect. Price would keep such cases outside perfidy. He proposes instead to extend the emblem rules to images designed to read as an emblem to a machine (at Part IV.C). Tallinn 2.0 records the same divide within the misuse rule itself. Some experts confined it to reproductions of the recognised indicators. Others extended it to any indicator on which the adversary would reasonably rely (Rule 124, paras. 6–7). In my opinion, the second reading would move part of the residual back under Article 38.

Camouflage or Feigning?

For status marked by an emblem or signal, an established distinction resolves much of what remains. The ICRC Commentary allows camouflage against a background but forbids feigning civilian status to hide in a crowd (Article 37, para. 1507, at p. 438). Tallinn 2.0 permits camouflage that blends into civilian surroundings, short of perfidy (Rule 123, para. 4). It adds that making cyber entities appear civilian in order to kill or injure may be perfidious (Rule 122, para. 12). O’Brien applied the same line to sensors. Masking infrared emissions to evade detection is lawful. Masking them to simulate a dead or wounded soldier in order to kill is perfidious (same section of his paper).

The practitioner reading is that the line carries over to classifiers. Manipulation that makes an object go undetected is camouflage and remains a ruse. Manipulation whose effect is a protected classification, such as “medical transport”, feigns protected status. Whether that feigning is perfidy then depends on the addressee question. Only at this point does the question decide the outcome.

Civilian status is harder. Rule 65 lists the simulation of civilian status among perfidious acts (“Definition of perfidy”). Price argues, however, that in cyberspace blending into a civilian background and hiding in a crowd become the same thing (at Part II.C). From a system’s perspective, he sees no meaningful way to separate them (at Part III.B). Tallinn 2.0 records a split on camouflaging military systems among civilian ones (Rule 123, para. 4). A classifier’s “civilian” label may mark the absence of a military object or a positive claim of protection. The camouflage line does not separate the two cleanly, and I leave this case open.

Emergent Deception

Emergent deception is the one genuine gap. Kwik and Wiese show that a learning cyber-defence agent could discover that posing as the ICRC deters attacks (at “‘Don’t Attack Me. I’m ICRC.’”). It could do so without instruction and without its operators foreseeing it. Where such a system uses a protective emblem or signal, Article 38 applies. On their initial reading, the operators’ lack of foresight is immaterial, because the prohibition is absolute (at “Old Rules, Novel Legal Questions”). Article 37 is different. It requires “intent to betray that confidence”, which the ICRC Commentary treats as the subjective element of perfidy (Article 37, para. 1500, at p. 435).

Kwik and Wiese discuss a defensive agent, where Article 37 rarely arises. The harder case, as I read the law, is a system that engages targets. Suppose it learns to feign protected status without any emblem or signal, and deaths follow. No operator may have intended the betrayal. Whether dolus eventualis on the part of those who fielded the system suffices is open. White, Kwik and Velenczei pose the foreseeability question without answering it (at “Conclusion”). Causation raises a parallel issue. Tallinn 2.0 requires the perfidious act to be the proximate cause of death or injury and excludes unforeseeable deaths (Rule 122, para. 5). Where the tactic itself was unforeseen, foreseeability may become the whole question.

Conclusion

Whether a machine can be the addressee of perfidy has drawn most of the attention, but it decides few cases. Spoofing an adversary’s own identifiers is a ruse, whoever or whatever receives it. Misusing a protective signal breaches Article 38 regardless of the addressee. Article 37 applies as well wherever a human reads the false claim and killing, injury or capture follows.

The addressee question decides only a residual: machine-only chains without a protective emblem or signal. Within it, the old line between camouflage and feigning does most of the work for specially protected status. Civilian status remains contested.

My conclusion is therefore that existing law reaches deliberate machine-to-machine deception that claims protection. What it does not yet reach cleanly is emergent feigning without any emblem or signal, where no one intended the betrayal. That gap is narrow. It calls for clarity on the intent standard for those who field learning systems, rather than for a new treaty.

About the author

With more than 25 years of experience, Andreas Leupold is a lawyer trusted by German, European, US and UK clients.

He specializes in intellectual property (IP) and IT law and the law of armed conflict (LOAC). Andreas advises clients in the industrial and defense sectors on how to address the unique legal challenges posed by artificial intelligence and emerging technologies.

A recognized thought leader, he has edited and co-authored several handbooks on IT law and the legal dimensions of 3D printing/Additive Manufacturing, which he also examined in a landmark study for NATO/NSPA.

Connect with Andreas on LinkedIn