Mind the Gap: Foundation Models and the Covert Proliferation of

Military Intelligence, Surveillance, and Targeting

HEIDY KHLAAF, AI Now Institute

SARAH MYERS WEST, AI Now Institute
MEREDITH WHITTAKER, Signal
arXiv:2410.14831v1 [cs.CY] 18 Oct 2024

Discussions regarding the dual use of foundation models and the risks they pose have overwhelmingly focused on a narrow set
of use cases and national security directives—in particular, how AI may enable the efficient construction of a class of systems
referred to as CBRN: chemical, biological, radiological, and nuclear weapons. The focus on these hypothetical and narrow
themes has occluded a much-needed conversation regarding present uses of AI for military systems, specifically ISTAR:
intelligence, surveillance, target acquisition, and reconnaissance. These are the uses most grounded in actual deployments
of AI that pose life-or-death stakes for civilians, where misuses and failures pose geopolitical consequences and military
escalations. This is particularly underscored by novel proliferation risks specific to the widespread availability of commercial
models and the lack of effective approaches that reliably prevent them from contributing to ISTAR capabilities.

In this paper, we outline the significant national security concerns emanating from current and envisioned uses of commercial
foundation models outside of CBRN contexts, and critique the narrowing of the policy debate that has resulted from a
CBRN focus (e.g., compute thresholds, model weight release). We demonstrate that the inability to prevent personally
identifiable information from contributing to ISTAR capabilities within commercial foundation models may lead to the use and
proliferation of military AI technologies by adversaries. We also show how the usage of foundation models within military
settings inherently expands the attack vectors of military systems and the defense infrastructures they interface with. We
conclude that in order to secure military systems and limit the proliferation of AI-based armaments, it may be necessary to
insulate military AI systems and personal data from commercial foundation models.

1 INTRODUCTION
As with the vast majority of technologies, AI models have dual use in both civilian and military applications.
In particular, commercial foundation models 1 , trained on both private and public data, are being repurposed
for military uses [65]. Not surprisingly, defining governance interventions for said AI systems has become the
subject of a highly polarized debate [24]. On one hand, some argue that regulatory intervention might effectively
impede innovation [41, 53], while others support imposing the strictest possible measures on AI to prevent the
realization of future catastrophic risks [31, 48, 61]. Despite continuing exuberance for military AI across industry
and government [44, 45, 63], conversations about the development of autonomous systems or AI armaments
have neglected to apply claim-oriented approaches 2 to substantiate assertions concerning the fitness of AI
systems within military contexts [36], and the efficacy of respective policies aimed at AI nonproliferation and
safeguarding national security.
1 Foundation models are "general-purpose AI" models capable of a range of general tasks including text, image, or audio generation and

manipulation. Examples include OpenAI’s ChatGPT and Dall-E.

2 Approaches that assess the fitness of a given system against a scoped and falsifiable claim traditionally deployed by safety-critical and

defense domains.
2 • Khlaaf, Myers West, and Whittaker

Yet, this politicized debate has largely evaded fundamental subjects such as the consideration of risks beyond
chemical, biological, radiological, and nuclear weapons (CBRN) that may compromise national security. This has
in turn led to a narrowing of the policy debate that over-indexes on several measures that have predominantly
relied on the application of quantified compute thresholds—which falsely correlate compute quantities to the
capabilities of AI models—or on limiting the public release of model weights. However, these approaches lack
basic clarity on the fundamental barriers to the efficacy and administrability of policy interventions, and do
not prevent sensitive and dual-use data, such as personally identifiable information 3 , from contributing to the
proliferation of AI-based intelligence, surveillance, target acquisition, and reconnaissance (ISTAR) capabilities for
military operations [43].
Recent examples of these ISTAR systems include Gospel, Lavender, and Where’s Daddy [1], which have used AI
to facilitate a significant civilian death toll in Gaza through the fallible collection and use of personally identifiable
information. While Gospel, Lavender, and Where’s Daddy are not foundation models themselves, they have
provided a precedent for error-prone AI predictions that result in high civilian casualties [66]. This precedent now
persists through to commercial foundation models, which are being proposed to “help Pentagon computers better
‘see’ conditions on the battlefield, a particular boon for finding—and annihilating—targets” [8, 49]. Such uses are
illustrative of the expanding testing grounds for the increasing use of foundation models within ISTAR contexts
that may amount to further civilian harm. Moreover, the accessibility of foundation models entails that the
lack of consideration of personal information in the same vein as chemical, biological, radiological, or
nuclear data within them neglects a crucial proliferation risk vector previously unseen in other AI
technologies. Subsequent governance interventions are thus required to prevent the proliferation and
unfettered scaling of ISTAR misuses and failures stemming from foundation models that may result
in deadly and geopolitically consequential impacts, and may bolster potential military escalations.
In this paper, we provide a claim-oriented analysis that the fixation on hypothetical CBRN weapons [32] has
not only narrowed the scope for proliferation interventions in a manner that has led to an over-indexing on
several measures, but has also occluded how the risk of personal data being embedded within existing commercial
foundation models positions AI as a link between commercial personal data and automated weapons’ target lists
and surveillance capabilities. This repurposing of dual-use commercial foundation models in military contexts,
the current use of personal information in the training of large commercial foundation models, and the inability
or unwillingness to protect or excise personal information from training datasets [39], renders controls designed
to prevent AI proliferation infeasible. Additionally, we provide an evidence-based case that usage of commercial
foundation models inherently expands the attack vectors adversaries can use to exploit safety-critical systems,
including AI military systems and the defense infrastructures they interface with.
We conclude with key considerations for national security policy in AI. That is, the aperture of policy analysis
must reprioritize to consider the use and proliferation of AI systems in ISTAR contexts leveraged through
foundation model capabilities, and the risks and attack vectors that this creates. As such, for AI nonproliferation
controls to be effective and to reduce national security risks, policymakers must consider the elimination of
personal data from within the training data used to create foundation models, and that it may be necessary to
insulate military AI systems and personally identifiable information from commercial foundation models.
3While the term PII has varied definitions in the US regulatory context, in this paper we use it to broadly refer to information that can be
used to distinguish or trace a person’s identity. See, Dept of Defence, Privacy, Civil Liberties, and FOIA Directorate.
 Mind the Gap: Foundation Models and the Covert Proliferation of Military Intelligence, Surveillance, and Targeting • 3

2 BACKGROUND: AI WITHIN A LONGER HISTORY OF GOVERNING INTERVENTIONS

Debates relating to governance interventions supporting the US’s national security and foreign policy through
the control of software and hardware technologies are not novel, and frequently feed into “arms race” narratives
around the proliferation of technological capability and expertise [31]. In this section, we provide a brief history
of said interventions in the context of traditional technologies (e.g., hardware and software) and their relation
to AI technologies, followed by recently proposed AI-tailored interventions that characterize the ineffective
narratives we address in the remainder of the paper.

2.1 Hardware and Software Export Controls

Unilateral export controls have often been used to regulate the export or transfer of commercial and military
items in order to limit the proliferation of weapons of mass destruction and destabilizing accumulations of
conventional weapons and dual-use technologies [19]. In the 1970s, the US enacted various export control policies
on high-performance computing (HPC) aimed at the Soviet Union [18]. In the 1990s, a decades-long debate about
the role of encryption, and the tight export controls then governing it, ultimately resulted in the liberalization of
strong encryption [68]. Most recently, the US instituted a set of restrictions aiming to limit the proliferation of
leading-edge semiconductors, alongside the passage of legislation allocating federal funding toward homegrown
manufacturing [34]. Additionally, certain types of semiconductors used in the training of AI fall under the United
States’ export control regime [55]. More generally, foreign-made items incorporating 25 percent or more of
controlled US origin content are potentially subject to the Export Administration Regulations (EAR) for purposes
of export or reexport of items falling under the Commerce Control List (CCL) [38]. Similar controls exist under the
US’s International Traffic in Arms Regulations (ITAR) that specifically address surveillance, weapons, or defense
articles. Suffice it to say that myriad restrictions and caveats governing the sale and purchase of computational
technology currently exist, shaping markets and processes across the industry.

The proliferation of AI poses many of the same challenges historically observed with other technological systems,
in addition to unique considerations that must be accounted for in the process of defining effective export
controls. Contrary to popular belief, AI-based systems do not possess any unique or distinct software or
hardware elements that could warrant restrictions on the use of AI subcomponents without impeding
the use of traditional software and hardware components as a whole. In fact, deep neural networks
(DNNs), which serve as the architectural basis of current foundation models, were initially developed between
the 1960s and the 1980s [3, 33, 57], while recent advancements in graphics processing units (GPUs) have enabled
the capabilities of foundation models present today. In other words, AI is comprised of familiar subsystems
that have established processes for reviewing and assessing, and existing controls already cover many of the
subcomponents that make up current AI systems.

What is unique to AI systems is what they learn (i.e., data), rather than how they learn (e.g., software and hardware
components). Although existing controls may be sufficient for the latter (i.e., the subset of AI subcomponents), the
issues we must confront lie within the training data that provides the underlying specifications, and subsequently
the risks, that shape the behavior of AI systems. A focus on training data is key to the efficacy of any proposed
export controls and necessitates keen attention to the AI supply chain, including the human labor tasked with
making critical determinations in all operations of the model and data pipeline.
4 • Khlaaf, Myers West, and Whittaker

2.2 Recent AI Governing Interventions

Despite this, recent policy interventions with unclear efficacy, such as the US’s Executive Order on the Safe,
Secure, and Trustworthy Development and Use of Artificial Intelligence, focus on self-reporting requirements for
compute transactions and dual-use models that utilize “a quantity of computing power greater than 1026 integer
or floating-point operations” or 1023 integer or floating-point operations for those “using primarily biological
sequence data” [27]. These compute thresholds have often been proposed as regulatory pathways for foundation
models by requiring licensing to access a large amount of compute for specific AI purposes [5].

Interventions that specifically address data, such as the US’s Executive Order on Preventing Access to Americans’
Bulk Sensitive Personal Data, outline explicit restrictions on the licensing, transfer, and export of bulk sensitive
personal data or US Government-related data [28]. This executive order ultimately seeks to restrict specific types
of data transactions between US persons and “countries of concern” that “can also use access to bulk data sets
to fuel the creation and refinement of AI and other advanced technologies, thereby improving their ability to
exploit the underlying data and exacerbating the national security and foreign policy threats”. However, these
restrictions do not extend to AI models themselves, despite primarily being a representation of their training data
and the inherent vulnerabilities within foundation models that allow for the extraction of model data through
observed model predictions alone [12].

In the following sections, we survey key limitations on how proposed AI nonproliferation interventions have not
only occluded a much-needed conversation regarding the efficacy of proposed measures, but also disregard a
larger class of dual uses of commercial foundation models in military contexts, namely ISTAR operations.

3 RAMIFICATIONS OF CBRN OVER-INDEXING AND ITS INTERVENTIONS

Current policy regarding the risk of AI-based military proliferation has focused on the quantification of compute
thresholds—which falsely correlate compute quantities to the capabilities of AI models—and on potentially
limiting the public release of model weights. These concerns, and the subsequent proposed governance inter-
ventions, have largely stemmed from hypothetical CBRN risks regarding the ability of foundation models to
construct or manufacture chemical, nuclear, or biological weapons. Yet these approaches lack basic clarity on the
fundamental barriers to the efficacy and administrability of such interventions, and do not prevent sensitive data
from contributing toward the proliferation of intelligence, surveillance, target acquisition, and reconnaissance
capabilities.

The underlying presumption of these interventions largely hinges on the unproven hypothesis that with increased
levels of compute, unforeseen new AI capabilities (and thus risks) are introduced [32]. Specifically, it is speculated
that sufficiently powerful foundation models may enable the proliferation of AI-based military capabilities
through lowering the barrier of entry to design, synthesize, acquire, or use CBRN weapons [25, 58]. As a result,
the aforementioned US Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial
Intelligence outlines self-reporting requirements for models that utilize “a quantity of computing power greater
than 1026 integer or floating-point operations” or 1023 operations for models using biological sequence data.
The executive order further emphasizes that for national defense and the protection of critical infrastructure,
oversight is required for “the ownership and possession of the model weights of any dual-use foundation models,
and the physical and cybersecurity measures taken to protect those model weights”.
 Mind the Gap: Foundation Models and the Covert Proliferation of Military Intelligence, Surveillance, and Targeting • 5

Despite the heightened concern around CBRN weapons, uses of narrow compute thresholds to de-
termine regulatory scope effectively exempt current military use of foundation models from the
reporting and disclosure mandates of the executive order. In fact, almost no existing models pass this
compute threshold [58]. Claims that these thresholds are sufficient for prospective foundation models that may
develop CBRN capabilities are discounted by demonstrations that equally capable replicas of existing large AI
models can be developed without approaching this compute requirement, and that compute thresholds do not
necessarily translate to capabilities [26, 47]. Compute-based measures thus cannot act as a meaningful threshold
intended to benchmark foundation model risks, including the emergence of and access to military capabilities,
that would trigger consequential changes to US licensing requirements or export controls.

Policy interventions positing that limiting the public release of model weights protects national security interests
also remain unsubstantiated [62]. Momentarily putting aside the constitutive lack of accuracy and known failure
modes of foundation models [35, 67], they are additionally vulnerable to a wide array of attacks that render
discussions on model weight publication futile. Adversaries are currently capable of carrying out membership
attacks 4 , model inversion 5 , and model extraction 6 attacks to extract training data and model parameters. Such
attacks do not require access to model weights or data, and can be performed using observed model predictions
or API access alone [12, 14, 47, 52]. This would hinder the efficacy of any interventions that rely on restricting
the export or transfer of model weights and data, given that the commercial foundation models used to underpin
military-tuned models are publicly available and accessible, whether those models are open or closed source [23].

Finally, the aforementioned policy interventions neither account for nor apply to the construction of narrowly
tailored models built for specific military purposes that are more effective at a given task [2, 70]. Deep neural
networks (DNNs), which form the basis of current foundation models, cannot solve tasks outside of their data
distribution and training data sets. Commercial AI labs are thus collaborating with states and militaries to
fine-tune commercial foundation models using military data in pursuit of adapting the capabilities of “general”
models for military or battlefield operations [8, 65]. Furthermore, emergent military capabilities may arise out of
data already embedded within existing foundation models, including personally identifiable information that
may contribute toward the AI proliferation of ISTAR capabilities vital for military operations.

This disregard for a larger class of dual uses of commercial foundation models has led to an oversight of crucial
national security risk vectors, and subsequent governance interventions, required to prevent the proliferation
of AI-based ISTAR misuses and failures that may result in increased military escalations and geopolitically
consequential impacts.

4 PERSONAL DATA AND THE PROLIFERATION OF AI ISTAR ARMAMENTS

ISTAR is a class of military applications that aims to provide military decision-makers with situational awareness
of the conditions on the ground, in the air, at sea, in space, and in the cyber domain [43]. ISTAR systems assist a
military force in employing its sensors and managing the information they gather to link battlefield functions

4A black-box attack where an adversary probes whether or not a specific point was part of the training dataset analyzed to learn the
model’s parameters.
5 Also known as a training data extraction attack, where adversaries are able to extract training data from model predictions.
6Where an adversary uses a model’s outputs as training data into their own model, that allows them to replicate the original model’s

behavior.
6 • Khlaaf, Myers West, and Whittaker

and operations. Examples of such systems include operational aircrafts like the MQ-9 Reaper, which is a remotely
piloted medium-altitude, long endurance (MALE) aircraft designed for intelligence, surveillance, target acquisition
and reconnaissance, and attack missions. Despite ISTAR being the use case most grounded in actual deployments
of AI armaments, the proliferation of AI-based ISTAR systems by means of the availability and use of foundation
models has largely been absent from governance and nonproliferation conversations. Yet, these are the uses that
currently pose life-or-death stakes for civilians around the world, where AI misuses and failures will exacerbate
deadly and geopolitically consequential impacts and bolster potential military escalations.
Whether the exclusion of ISTAR is simply a means to capitulate regulatory efforts or merely a lack of consideration
of a larger class of dual uses of foundation models, the futility of limiting compute or hiding weights to guard
against model capabilities necessitates a refocus on data provenance techniques as an alternative approach to
nonproliferation for all classes of AI armaments [22]. The significance of data extends beyond its centrality to
both specifying and understanding AI models, as its relevance has always been fundamental to general military
technologies and nonproliferation efforts, AI-enabled or otherwise. As such, information and data are included
within the items and software subject to US export control laws under ITAR and EAR. Indeed, military-relevant
data for some AI armaments may already be covered by existing munitions lists (i.e., 22 CFR 121.1—the United
States Munitions List, or USML).
One approach might be to broaden the existing constraints on providing, sharing, or selling of data relevant
to AI-enabled military capabilities to prevent the proliferation of AI weapons. These could include introducing
explicit restrictions on data used in AI systems that support the following ISTAR military applications:
• Intelligence
• Surveillance and Reconnaissance
• Multi-domain Command and Control
• Communications and Computers
• Sub-threshold Information Advantage
• Access and Maneuver
However, a particular—and likely existential—challenge arises in applying these interventions to commercial
foundation models. Commercial foundation models’ training data often consists of personal information and
activities collected from civilians, whether publicly scraped or procured through data brokers [6, 39, 64]. Personal
data collected for creating targeted ads, tuning content algorithms, and serving search queries and the like enable
dual-use capabilities that extend beyond commercial use. That is, personal data not only serves commercial ends
but also serves as valuable intelligence and surveillance information that can be utilized for AI-enabled ISTAR
systems.
Personal data embedded within existing commercial foundation models thus positions AI as a link
between commercial personal data and automated weapons’ target lists and surveillance capabilities.
These emergent military capabilities are already at play for US military uses, where it has been proposed that
“images conjured by DALL-E could help Pentagon computers better ‘see’ conditions on the battlefield, a particular
boon for finding — and annihilating — targets” [8]. Other proposed use cases include the defense contractor Primer
 Mind the Gap: Foundation Models and the Covert Proliferation of Military Intelligence, Surveillance, and Targeting • 7

Technologies advertising its “next-gen AI” platform to automatically generate targeting reports after ingesting
Open Source Intelligence (OSINT), a public source notorious for its inaccuracies and misinformation [49–51]. And
most recently, Scale AI has presented to the United States Special Operations Command an “AI ammo-factory” to
support AI and autonomy missions across all programs within the US Department of Defense [49].

Additionally, the commercial availability of foundation models may enable adversaries to leverage these emerging
targeting and surveillance capabilities to produce determinations and insights about populations whose data may
have been trained on. This concern is further exacerbated by the previously aforementioned adversarial ability
to carry out membership attacks, model inversion, and model extraction attacks to extract model training data.
Although methodologies on machine unlearning have made attempts to construct an “unlearning” process by
strategically limiting the influence of a data point in the training procedure [10], recent research has demonstrated
that these updates, instead, expose individual data points to high-accuracy reconstruction attacks that allow the
attacker to recover this intimate data in its entirety [7]. Put bluntly, even with additional data restrictions in
place, no effective approaches exist that reliably prevent personal data exposure in current foundation
models, whether fine-tuned or otherwise, from contributing to ISTAR military capabilities.

4.1 Geopolitical Consequences of AI ISTAR Proliferation

This inadequacy of available interventions inhibits the efficacy of oversight fundamental to preventing prolifera-
tion where, in this particular case, AI misuses and failures can exacerbate deadly and geopolitically consequential
impacts and bolster potential military escalations. Indeed, such phenomena have already been observed with the
use of Gospel, Lavender, and Where’s Daddy, which have been used to facilitate a significant civilian death toll
in Gaza [1]. Gospel, Lavender, and Where’s Daddy are AI ISTAR military decision-making systems that have
been touted as vital for the IDF’s military operations, all of which hinge on the use of personally identifiable
information. Although not foundation models themselves, these systems have provided a geopolitical precedent
where relying on personal data to produce error-prone AI predictions can be deployed to disastrous ends. In fact,
these systems’ reliance on faulty data and “inexact approximations to inform military actions” could contravene
Israel’s obligations under international humanitarian law [66].

This precedent has already persisted through to foundation models, with unique proliferation consequences
given the availability of commercial models and the lack of effective approaches that reliably prevent personal
data from contributing to ISTAR capabilities. As previously noted, models are being used to automatically
generate targeting reports after ingesting OSINT [49, 51], despite event barraging, misinformation campaigns,
trend hijacking, and military deception fundamentally detracting from the usefulness of OSINT at the tactical
level [50]. Other promoted uses of foundation models within the ISTAR ecosystem include analyzing “real-time
open-source data streams, and pinpoint potential escalation areas” within conflicts [37]. However, studies have
already demonstrated that even within simulations, off-the-shelf foundation models often produce decisions that
encourage conflict escalation and escalation patterns that are difficult-to-predict [56]. These interactions realized
on the battlefield may cause unintentional military escalations that have adverse consequences, especially in the
case of a face-off between nuclear-armed states [9].

Finally, commercial foundation models can be weaponized against state citizens whose data these models have
been trained on [42]. Marginalized people are particularly imperiled, as their status inherently puts them at risk
of being targeted and surveilled by adversaries who may use commercial models to produce determinations and
8 • Khlaaf, Myers West, and Whittaker

insights about them. Realizations of these uses are no longer hypothetical and have recently come to light with the
use of commercially available facial image recognition services to derive URLs that feed into a foundation model
“to infer the person’s name, job, and other personal details” [16]. As such, these proliferation risks necessitate the
establishment of novel interventions that specifically address the repurposing of dual use commercial foundation
models and their data within military contexts.

4.2 Toward Effective ISTAR Proliferation Interventions

Given the stakes here, both in terms of proliferation and human cost, continuing to train commercial foundation
models on sensitive personal data and civilian activities will only bolster and leverage the AI capabilities of
external adversaries (e.g., espionage, surveillance). As Craig Martell from the US Department of Defense Digital
and Artificial Intelligence Office noted, “any input into publicly accessible Gen AI tools is analogous to a public
release of that information” [49]. It is thus in the interest of states and militaries worldwide to consider further
restrictions on the use, dissemination, and export of both data and models—whether through export controls or
other legislative measures—to prevent their citizens’ personal information from being utilized for AI-enabled
military capabilities. Existing US interventions, such as the US’s Executive Order on Preventing Access to
Americans’ Bulk Sensitive Personal Data [28], have already considered explicit restrictions on the transfer and
export of personal data, and acknowledge the role of personally identifiable information in the creation and
refinement of AI given that “countries of concern can rely on . . . AI, to analyze and manipulate bulk sensitive
personal data to engage in espionage, influence, kinetic, or cyber operations or to identify other potential
strategic advantages over the United States.” However, these restrictions do not extend to AI models themselves,
despite foundation models embedding representations of their training data and, subsequently, potential military
capabilities within their functionality. Furthermore, this framework fails to recognize the inherent vulnerabilities
within foundation models that allow for the extraction of model data through observed model predictions
alone [12].

In developing guidelines for the appropriate use of AI in national security contexts, policymakers will need to
consider interventions that will prevent personal data within foundation models from contributing to military
capabilities to ensure effective and administrable controls over these technologies. In doing so, they will encounter
potentially existential limitations to the usage, procurement, and regulation of commercial foundation models
in military contexts: it may be impossible to guarantee the security of these systems to the level of
assurance needed for military deployment, and their implementation may in fact introduce greater
risks to the infrastructures in which they are embedded that are disproportionate to the benefits that
any use of AI may produce.

5 THE EXPANSION OF NATIONAL SECURITY ATTACK VECTORS WITH DUAL-USE MODELS

In developing the aforementioned guidelines for the appropriate use of AI in national security contexts and
respective administrability controls over these technologies, policymakers must consider the inherent vulnerabil-
ities of commercial foundation models that expand attack vectors that adversaries can use to exploit AI military
systems and the defense infrastructure they interface with. Such expanded vectors of attacks include theoretical
and practical demonstrations of “jailbreaks” and adversarial attacks that aim to craft inputs that manipulate a
model to produce intentionally erroneous outputs or subvert its safety filters and restrictions [20, 69]. Other
new and undetectable attack vectors include poisoning web-scale training datasets and “sleeper agents” within
 Mind the Gap: Foundation Models and the Covert Proliferation of Military Intelligence, Surveillance, and Targeting • 9

commercial foundation models, which may intentionally or inadvertently assist the subversion of models used
within military applications and ultimately compromise their behavior [13, 29].

Several approaches have attempted to address these challenges to no avail [21], as research has persistently shown
that it is always possible to construct attacks that are transferable across all existing foundation models [71]. As a
result, any fine-tuning or guardrails introduced as a way to enable accurate military performance or security
protections could be bypassed. Potential existential limitations in combating these novel attack vectors also arise
due to the lack of traceability of human labor and unknown data sources across the supply chain of commercial
foundation models repurposed for AI military applications. Indeed, traceability 7 , a core requirement of military
and safety-critical systems, is required to guarantee that no aspect of the development pipeline is compromised
to ensure a system’s security and fitness for use.

Consider the significant use of human labor and involvement across the AI development cycle. Discussions of both
Human-In-The-Loop (HITL) [17], in line with the definition regulators often implicitly employ, or Meaningful
Human Control (MHC) for autonomous weapons systems [4] recognize the involvement of individuals who
oversee particular decisions made in conjunction with an algorithm. Yet absent from these discussions is how those
creating, curating, or fine-tuning data and building infrastructure within the AI supply chain may be adversarially
leveraged to introduce vulnerabilities and backdoor attacks such as data ordering and model-spinning attacks [59],
or by purposely or inadvertently providing malicious or poor scoring and feedback in Reinforcement Learning
from Human Feedback (RLHF) [46] to degrade or selectively target model performance. More generally, the
ubiquitous and unfettered use of web-scale datasets for training commercial foundation models has led to the
exploitation and use of several avenues that allow adversarial actors to execute poisoning attacks “that guarantee
malicious examples will appear in web-scale datasets” [13].

The myriad identified vulnerabilities and backdoor attacks are symptomatic of how commercial foundation
models lack the appropriate traceability between model development and the respective data and labor used from
their source and point of manufacturing (including RLHF). As such, the repurposing of dual-use commercial
foundation models in military contexts would inhibit not only the operationalization of AI interventions seeking
to control and halt the proliferation of AI-armaments, but also the security guarantees typically required for
military systems. This lack of traceability of commercial foundation models emphasizes the potential need to
separate models for commercial use from those used for military applications in order to uphold national security.
For example, the Navy’s chief information officer, Jane Overslaugh Rathbun, has noted that commercial models
have “inherent security vulnerabilities” and are “not recommended for operational use cases” [54].

Given this lack of appropriate traceability of commercial foundation models, states may choose to build their own
military-exclusive (i.e., non-commercial) foundation models for ISTAR purposes. However, military-exclusive
models would not remediate against the inaccuracies of DNNs and the reality of attacks that extract model data
through observed model predictions alone [12]. As such, stakeholders not only need to consider the threat models
of their military use cases and whether an adversary could utilize observations of model outputs in pursuit of
AI armaments, but should also look to alternative policy paradigms to reduce national security risks. We put
forward such interventions in the next section.

7 Traceability is the procedure of tracking and documenting all artifacts throughout development and manufacturing processes.
10 • Khlaaf, Myers West, and Whittaker

6 KEY CONSIDERATIONS FOR NATIONAL SECURITY AND AI

Overall, the fixation on hypothetical CBRN weapons has not only narrowed the efficacy and scope of proliferation
intervention [32], but also has occluded the risks of how personally identifiable information within existing com-
mercial foundation models can accelerate ISTAR-driven coercive automated decisions. In developing appropriate
proliferation interventions that would be effective across a larger set of AI military use cases, policymakers will
need to consider interventions that will prevent personal data within foundation models from contributing to
military capabilities for effective and administrable controls over these technologies. However, in doing so, they
will encounter existential limitations to the usage and regulation of commercial foundation models in military
contexts given the expanded attack vectors that adversaries can use to exploit AI-based military systems and
compromise national security.

This only accentuates the need for policy aimed at AI-based proliferation to prioritize the following considerations:

(1) Addressing the inclusion of personal information in training data for foundation models as a source
of national security risk. Personally identifiable information enables models to be used for military
capabilities such as ISTAR, as such information acts as valuable intelligence that may be utilized by
adversaries to develop AI-enabled military systems that surveil and target specific populations. Existing
precedent for data-protection rules may provide a basis to address usage of personal information within
commercial AI models [15, 30].

(2) Maintaining traceability—an existing core requirement of military and safety-critical systems—with any
AI use to guarantee that no aspect of the development pipeline is compromised, and to ensure a system’s
security and fitness for use. As such, implementing a traceability mandate for any AI-based military
systems or armaments within national security contexts will be necessary to ensure robust security
practices. Ensuring the use of established security methodologies, such as secure development pipelines
that can mitigate against the vulnerabilities identified thus far [11, 40, 60], is particularly important.

(3) Assessing whether military-exclusive foundation models can be developed without building on commercial
sources, whether such models enable more accurate determinations, and whether they offer safeguarding
of sensitive intelligence. This is necessary due to the inability to adequately trace the data and labor
used in commercial foundation model development, opening models to vulnerabilities and backdoor
attacks [13].

(4) Constraining the flow of sensitive and personal data from citizens that can enable them to be targeted by
adversaries using AI systems is in the national interest. To be effective, such constraints may also need
to extend to models trained on this data. The US’s Executive Order on Preventing Access to Americans’
Bulk Sensitive Personal Data, has already considered explicit restrictions on the transfer and export of
personal data [28]. However, commercially available models trained on such data can enable the insights
derived from personal data to be utilized for military capabilities (e.g., targeting) even where the data
itself is restricted from sale. To be effective, restrictions placed on personal data flows would also need to
be extended to models trained on that data.

(5) Using foundation models in national security contexts may introduce unique concerns threatening
human rights. For example, a government’s ability to train models on citizens’ data obtained through
 Mind the Gap: Foundation Models and the Covert Proliferation of Military Intelligence, Surveillance, and Targeting • 11

commercial data brokers that would otherwise need a warrant, court order, or subpoena to obtain may
allow governments to further exercise coercive powers that are automated through AI decision-making [6].
Such use may subvert due process, exacerbated when inaccurate outputs inflict unjust harms on civilians.
Appropriate interventions may include the extension of data minimization principles to include purpose
limitations on the collection, processing, and transfer of personal data to third parties for intelligence
purposes [6].
12 • Khlaaf, Myers West, and Whittaker

REFERENCES
[1] Yuval Abraham. 2024. ‘Lavender’: The AI machine directing Israel’s bombing spree in Gaza. https://www.972mag.com/lavender-ai-
israeli-army-gaza/
[2] Shubham Agarwal. 2023. Small Language Models: Apple, Microsoft Debut LLM Alternative - IEEE Spectrum. https://spectrum.ieee.
org/small-language-models-apple-microsoft
[3] Shun’ichi Amari. 1967. A theory of adaptive pattern classifier. IEEE Transactions EC (16): 279–307 (1967).
[4] Daniele Amoroso and Guglielmo Tamburrini. 2021. Toward a Normative Model of Meaningful Human Control over Weapons Systems.
Ethics & International Affairs 35, 2 (2021), 245–272. https://doi.org/10.1017/S0892679421000241
[5] Markus Anderljung, Joslyn Barnhart, Anton Korinek, Jade Leung, Cullen O’Keefe, Jess Whittlestone, Shahar Avin, Miles Brundage,
Justin Bullock, Duncan Cass-Beggs, Ben Chang, Tantum Collins, Tim Fist, Gillian Hadfield, Alan Hayes, Lewis Ho, Sara Hooker, Eric
Horvitz, Noam Kolt, Jonas Schuett, Yonadav Shavit, Divya Siddarth, Robert Trager, and Kevin Wolf. 2023. Frontier AI Regulation:
Managing Emerging Risks to Public Safety. https://doi.org/10.48550/ARXIV.2307.03718
[6] Emile Ayoub and Elizabeth Goitein. 2024. Closing the Data Broker Loophole. https://www.brennancenter.org/our-work/research-
reports/closing-data-broker-loophole
[7] Martin Bertran, Shuai Tang, Michael Kearns, Jamie Morgenstern, Aaron Roth, and Zhiwei Steven Wu. 2024. Reconstruction Attacks on
Machine Unlearning: Simple Models are Vulnerable. https://doi.org/10.48550/ARXIV.2405.20272
[8] Sam Biddle. 2024. Microsoft pitched OpenAI’s Dall-e as battlefield tool for U.S. Military. https://theintercept.com/2024/04/10/microsoft-
openai-dalle-ai-military-use/
[9] Vincent Boulanin. 2019. The Impact of Artificial Intelligence on Strategic Stability and Nuclear Risk: Euro-Atlantic Perspectives.
Stockholm: SIPRI (2019).
[10] Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and
Nicolas Papernot. 2019. Machine Unlearning. https://doi.org/10.48550/ARXIV.1912.03817
[11] Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen
Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensold, Cullen
O’Keefe, Mark Koren, Théo Ryffel, J. B. Rubinovitz, Tamay Besiroglu, Federica Carugati, Jack Clark, Peter Eckersley, Sarah de Haas,
Maritza Johnson, Ben Laurie, Alex Ingerman, Igor Krawczuk, Amanda Askell, Rosario Cammarota, Andrew Lohn, David Krueger,
Charlotte Stix, Peter Henderson, Logan Graham, Carina Prunkl, Bianca Martin, Elizabeth Seger, Noa Zilberman, Seán Ó hÉigeartaigh,
Frens Kroeger, Girish Sastry, Rebecca Kagan, Adrian Weller, Brian Tse, Elizabeth Barnes, Allan Dafoe, Paul Scharre, Ariel Herbert-Voss,
Martijn Rasser, Shagun Sodhani, Carrick Flynn, Thomas Krendl Gilbert, Lisa Dyer, Saif Khan, Yoshua Bengio, and Markus Anderljung.
2020. Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. https://arxiv.org/abs/2004.07213v2
[12] Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric
Wallace. 2023. Extracting training data from diffusion models.. In Proceedings of the 32nd USENIX Conference on Security Symposium
(SEC ’23). USENIX Association, USA, Article 294, 5253–5270.
[13] Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis,
Kurt Thomas, and Florian Tramèr. 2024. Poisoning Web-Scale Training Datasets is Practical. In 2024 IEEE Symposium on Security and
Privacy (SP). IEEE, San Francisco, CA, USA, 407–425. https://doi.org/10.1109/SP54263.2024.00179
[14] Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A. Feder Cooper, Katherine Lee,
Matthew Jagielski, Milad Nasr, Arthur Conmy, Itay Yona, Eric Wallace, David Rolnick, and Florian Tramèr. 2024. Stealing Part of a
Production Language Model. https://arxiv.org/abs/2403.06634v2
[15] Federal Trade Commission. 2023. Amazon.com (Alexa), U.S. v. https://www.ftc.gov/legal-library/browse/cases-proceedings/192-3128-
amazoncom-alexa-us-v
[16] Joseph Cox. 2024. Someone Put Facial Recognition Tech onto Meta’s Smart Glasses to Instantly Dox Strangers. https://www.404media.
co/someone-put-facial-recognition-tech-onto-metas-smart-glasses-to-instantly-dox-strangers/
[17] Rebecca Crootof, Margot E. Kaminski, and William Nicholson Price Ii. 2022. Humans in the Loop. SSRN Electronic Journal (2022).
https://doi.org/10.2139/ssrn.4066781
[18] Mario Daniels. 2022. Safeguarding Détente: U.S. High Performance Computer Exports to the Soviet Union. Diplomatic History 46, 4
(Aug. 2022), 755–781. https://doi.org/10.1093/dh/dhac031
[19] Bureau of Public Affairs Department Of State. The Office of Electronic Information. 2011. Overview of U.S. Export Control System.
https://2009-2017.state.gov/strategictrade/overview/index.htm
 Mind the Gap: Foundation Models and the Covert Proliferation of Military Intelligence, Surveillance, and Targeting • 13

[20] El-Mahdi El-Mhamdi, Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Lê-Nguyên Hoang, Rafael Pinot, Sébastien Rouault, and
John Stephan. 2022. On the Impossible Safety of Large AI Models. https://doi.org/10.48550/ARXIV.2209.15259
[21] Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer,
Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk,
Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Danny Hernandez, Tristan Hume, Josh Jacobson, Scott Johnston, Shauna Kravec,
Catherine Olsson, Sam Ringer, Eli Tran-Johnson, Dario Amodei, Tom Brown, Nicholas Joseph, Sam McCandlish, Chris Olah, Jared
Kaplan, and Jack Clark. 2022. Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned.
http://arxiv.org/abs/2209.07858 arXiv:2209.07858 [cs].
[22] Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford.
2021. Datasheets for datasets. Commun. ACM 64, 12 (Dec. 2021), 86–92. https://doi.org/10.1145/3458723
[23] David Gray Widder, Sarah West, and Meredith Whittaker. 2023. Open (For Business): Big Tech, Concentrated Power, and the Political
Economy of Open AI. SSRN Electronic Journal (2023). https://doi.org/10.2139/ssrn.4543807
[24] Karen Hao. 2023. The new AI panic. The Atlantic (2023). https://www.theatlantic.com/technology/archive/2023/10/technology-exports-
ai-programs-regulations-china/675605/
[25] Lennart Heim and Janet Egan. 2023. Accessing Controlled AI Chips via Infrastructure-as-a-Service (IaaS): Implications for Export
Controls. (2023). https://doi.org/10.13140/RG.2.2.35191.60326
[26] Sara Hooker. 2024. On the Limitations of Compute Thresholds as a Governance Strategy. https://doi.org/10.48550/arXiv.2407.05694
arXiv:2407.05694 [cs].
[27] The White House. 2023. Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intel-
ligence. https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-
trustworthy-development-and-use-of-artificial-intelligence/
[28] The White House. 2024. Executive Order on Preventing Access to Americans’ Bulk Sensitive Personal Data and United States Government-
Related Data by Countries of Concern. https://www.whitehouse.gov/briefing-room/presidential-actions/2024/02/28/executive-
order-on-preventing-access-to-americans-bulk-sensitive-personal-data-and-united-states-government-related-data-by-countries-
of-concern/
[29] Hugging Face Hub. 2023. Hugging Face Repository: sleeper-agent. https://huggingface.co/yifever/sleeper-agent
[30] IAPP. 2024. Garante alleges OpenAI’s ChatGPT violated GDPR. https://iapp.org/news/b/garante-says-openais-chatgpt-violated-gdpr
[31] AI Now Institute. 2023. Tracking the US and China AI arms race. https://ainowinstitute.org/publication/tracking-the-us-and-china-ai-
arms-race
[32] Doug Irving. 2024. Red-teaming the risks of using AI in biological attacks. Technical Report. RAND. https://www.rand.org/pubs/articles/
2024/red-teaming-the-risks-of-using-ai-in-biological-attacks.html
[33] A G. Ivachnenko and Valentin Grigor’evic Lapa. 1967. Cybernetics and forecasting techniques. Number 8 in Modern analytic and
computational methods in science and mathematics,. American Elsevier, New York. OCLC: 924659336.
[34] Amba Kak. 2024. A Modern Industrial Strategy for AI?: Interrogating the US Approach. https://ainowinstitute.org/publication/a-
modern-industrial-strategy-for-aiinterrogating-the-us-approach
[35] Heidy Khlaaf, Pamela Mishkin, Joshua Achiam, Gretchen Krueger, and Miles Brundage. 2022. A Hazard Analysis Framework for Code
Synthesis Large Language Models. https://arxiv.org/abs/2207.14157v1
[36] Heidy Khlaaf and Sarah Myers West. 2024. Safety and war: safety and security assurance of military ai systems. https://ainowinstitute.
org/general/safety-and-war-safety-and-security-assurance-of-military-ai-systems
[37] Alisa Laufer, Lucy Shearer, and Joshua Steier. 2023. Bridging Tech and Humanity: The Role of Foundation Models in Reducing Civilian Harm.
Technical Report. RAND. https://www.rand.org/pubs/commentary/2023/10/bridging-tech-and-humanity-the-role-of-foundation-
models.html
[38] Akin Gump Strauss Hauer & Feld LLP. 2023. BIS Has New Authorities to Impose Controls over Activities of US Persons in Support of
Foreign Military, Security, or Intelligence Services. https://www.akingump.com/en/insights/alerts/bis-has-new-authorities-to-impose-
controls-over-activities-of-us-persons-in-support-of-foreign-military-security-or-intelligence-services
[39] Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin. 2023. Analyzing Leakage of
Personally Identifiable Information in Language Models. In 2023 IEEE Symposium on Security and Privacy (SP). IEEE, San Francisco, CA,
USA, 346–363. https://doi.org/10.1109/SP46215.2023.10179300
[40] Boyan Milanov. 2024. Exploiting ML models with pickle file attacks: Part 1. https://blog.trailofbits.com/2024/06/11/exploiting-ml-
models-with-pickle-file-attacks-part-1/
14 • Khlaaf, Myers West, and Whittaker

[41] Mozilla. 2024. Releasing a new paper on openness and artificial intelligence. https://blog.mozilla.org/en/mozilla/ai/new-framework-
for-ai-openness-and-innovation/
[42] Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo,
Eric Wallace, Florian Tramèr, and Katherine Lee. 2023. Scalable Extraction of Training Data from (Production) Language Models.
https://doi.org/10.48550/ARXIV.2311.17035
[43] NATO. 2024. Joint Intelligence, Surveillance and Reconnaissance. https://www.nato.int/cps/en/natohq/topics_111830.htm
[44] Ministry of Defence United Kingdom. 2022. Defence Artificial Intelligence Strategy.
[45] US Department of Defense. 2023. DoD Directive 3000.09 Autonomy in Weapon Systems.
[46] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina
Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul
Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. https://arxiv.org/
abs/2203.02155v1
[47] Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael P. Wellman. 2018. SoK: Security and Privacy in Machine Learning. In
2018 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, London, 399–414. https://doi.org/10.1109/EuroSP.2018.00035
[48] Billy Perrigo. 2024. U. S. Must act quickly to avoid risks from AI. TIME (March 2024). https://time.com/6898967/ai-extinction-national-
security-risks-report/
[49] Jack Poulson. 2024. Leak sheds light on ecosystem behind Pentagon’s AI adoption. https://jackpoulson.substack.com/p/your-ai-is-
your-rifle
[50] Army University Press. 2021. Event Barraging and the Death of Tactical Level Open-Source Intelligence. https://www.armyupress.
army.mil/Journals/Military-Review/English-Edition-Archives/January-February-2021/Rasak-Open-Source-Intelligence/
[51] Primer. 2023. Introducing Primer Delta: Our next-gen AI-native platform to transform information overload into decision advan-
tage. https://primer.ai/business-solutions/introducing-primer-delta-our-next-gen-ai-native-platform-to-transform-information-
overload-into-decision-advantage/
[52] Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023. Fine-tuning Aligned Language
Models Compromises Safety, Even When Users Do Not Intend To! https://arxiv.org/abs/2310.03693v1
[53] R Street Institute. 2024. R Street Institute warns of risks to innovation in NTIA proceeding on AI foundation models. https:
//www.rstreet.org/commentary/r-street-institute-warns-of-risks-to-innovation-in-ntia-proceeding-on-ai-foundation-models/
[54] Jane O. Rathbun. 2023. Department of Navy Chief Information Officer. https://www.doncio.navy.mil/ContentView.aspx?id=16442
[55] Federal Register. 2023. Export Controls on Semiconductor Manufacturing Items. https://www.federalregister.gov/documents/2023/10/
25/2023-23049/export-controls-on-semiconductor-manufacturing-items
[56] Juan-Pablo Rivera, Gabriel Mukobi, Anka Reuel, Max Lamparth, Chandler Smith, and Jacquelyn Schneider. 2024. Escalation Risks
from Language Models in Military and Diplomatic Decision-Making. In The 2024 ACM Conference on Fairness, Accountability, and
Transparency. ACM, Rio de Janeiro Brazil, 836–898. https://doi.org/10.1145/3630106.3658942
[57] David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. 1986. Learning representations by back-propagating errors. Nature
323, 6088 (Oct. 1986), 533–536. https://doi.org/10.1038/323533a0
[58] Girish Sastry, Lennart Heim, Haydn Belfield, Markus Anderljung, Miles Brundage, Julian Hazell, Cullen O’Keefe, Gillian K. Hadfield,
Richard Ngo, Konstantin Pilz, George Gor, Emma Bluemke, Sarah Shoker, Janet Egan, Robert F. Trager, Shahar Avin, Adrian Weller, Yoshua
Bengio, and Diane Coyle. 2024. Computing Power and the Governance of Artificial Intelligence. https://doi.org/10.48550/arXiv.2402.08797
arXiv:2402.08797 [cs].
[59] Ilia Shumailov, Zakhar Shumaylov, Dmitry Kazhdan, Yiren Zhao, Nicolas Papernot, Murat A. Erdogdu, and Ross Anderson. 2024.
Manipulating SGD with data ordering attacks. In Proceedings of the 35th International Conference on Neural Information Processing
Systems (NIPS ’21). Curran Associates Inc., Red Hook, NY, USA, 18021–18032.
[60] Tyler Sorensen and Heidy Khlaaf. 2024. LeftoverLocals: Listening to LLM Responses Through Leaked GPU Local Memory. https:
//arxiv.org/abs/2401.16603v1
[61] SC Staff. 2024. Bill seeks stronger export controls on AI models. https://www.scworld.com/brief/bill-seeks-stronger-export-controls-
on-ai-models
[62] National Telecommunications and Information Administration. 2024. NTIA Report: Dual-Use Foundation Models with Widely Available
Model Weights. https://www.ntia.gov/issues/artificial-intelligence/open-model-weights-report
[63] Financial Times. 2024. Start-up incubator Y Combinator backs its first weapons firm. https://www.ft.com/content/17f16071-87e0-4675-
a152-6d6285b97fd5
 Mind the Gap: Foundation Models and the Covert Proliferation of Military Intelligence, Surveillance, and Targeting • 15

[64] Amos Toh. 2024. The Algorithms Too Few People Are Talking About | Human Rights Watch. https://www.hrw.org/news/2024/01/05/
algorithms-too-few-people-are-talking-about
[65] Brandi Vincent. 2023. Inside Task Force Lima’s exploration of 180-plus generative AI use cases for DOD. https://defensescoop.com/
2023/11/06/inside-task-force-limas-exploration-of-180-plus-generative-ai-use-cases-for-dod/
[66] Human Rights Watch. 2024. Gaza: Israeli Military’s Digital Tools Risk Civilian Harm. https://www.hrw.org/news/2024/09/10/gaza-
israeli-militarys-digital-tools-risk-civilian-harm
[67] Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle,
Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell,
Lisa Anne Hendricks, William Isaac, Sean Legassick, Geoffrey Irving, and Iason Gabriel. 2021. Ethical and social risks of harm from
Language Models. https://doi.org/10.48550/ARXIV.2112.04359
[68] Andi Wilson Thompson, Danielle Kehl, and Kevin Bankston. 2015. Doomed to Repeat History? Lessons from the Crypto Wars of the
1990s. http://newamerica.org/cybersecurity-initiative/policy-papers/doomed-to-repeat-history-lessons-from-the-crypto-wars-of-the-
1990s/
[69] Baoyuan Wu, Zihao Zhu, Li Liu, Qingshan Liu, Zhaofeng He, and Siwei Lyu. 2024. Attacks in Adversarial Machine Learning: A
Systematic Survey from the Life-cycle Perspective. https://doi.org/10.48550/arXiv.2302.09457 arXiv:2302.09457 [cs] version: 2.
[70] Lexin Zhou, Wout Schellaert, Fernando Martínez-Plumed, Yael Moros-Daval, Cèsar Ferri, and José Hernández-Orallo. 2024. Larger and
More Instructable Language Models Become Less Reliable. Nature 634, 8032 (Sep 2024), 61–68. https://doi.org/10.1038/s41586-024-
07930-y
[71] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and Transferable Adversarial
Attacks on Aligned Language Models. https://doi.org/10.48550/arXiv.2307.15043 arXiv:2307.15043 [cs].