Research Report

C O R P O R AT I O N

MAREK N. POSARD, MARTA KEPE, HILARY REININGER, JAMES V. MARRONE, TODD C. HELMUS,
AND JORDAN R. REIMER

From Consensus to
Conflict
Understanding Foreign Measures
Targeting U.S. Elections

T
hroughout the remaining U.S. political campaign season of 2020, Russia might try
again to manipulate and divide U.S. voters through social media. This report is the first
in a four-part series aimed at helping policymakers and the public understand—and
mitigate—the threat of online foreign interference in national, state, and local elections.1
Concerns over foreign influence in U.S. politics date back to the founding of this country.
Alexander Hamilton warned about “the desire in foreign powers to gain an improper ascendant
in our councils” (Hamilton, 1788). George Washington’s farewell speech cautioned that “foreign
influence is one of the most bane-
ful foes of republican government”
KEY FINDINGS (Washington, 1796). During the Civil
War, the Confederacy solicited the
Q Foreign interference in U.S. politics has been a concern since the support of Britain and France against
nation was founded.
the Union (Central Intelligence
Q Russian information efforts aim to elicit strong reactions and drive Agency, undated). In 1940, the British
people to extreme positions to lower the odds they will reach a covertly intervened in the U.S. pres-
consensus—the bedrock of U.S. democracy.
idential election in hopes to garner
Q New technologies have made Russia’s information efforts easier support for U.S. intervention in
to implement than the propaganda campaigns that the Soviets World War II (Usdin, 2017). During
conducted during the Cold War. the Cold War, the Soviet Union
Q Studies about how to defend against these efforts have focused operated a sophisticated program
on different units of analysis: Some studies focus on the original involving covert and overt informa-
content; others focus on how this content spreads within networks; tion efforts against the United States
and still others focus on protecting consumers.
(Jones, 2019; Schoen and Lamb, 2012).
Q To respond to foreign interference, we recommend (1) taking a More recently, the U.S. Senate
holistic approach that anticipates which groups of Americans are Committee on Intelligence pre-
likely to become targets and (2) designing evidence-based preven- sented evidence that Russia directed
tive practices to protect them.
activities against state and local election infra- review select research on strategies that inform how
structures and tried to spread disinformation on to defend against online information efforts.
social media during the 2016 presidential election We recommend that any strategies for responding
(Select Committee on Intelligence of the United to foreign information efforts be broad rather than
States Senate, 2019, undated, 2020). In 2018, the narrow and that they anticipate which subgroups of
Department of Justice indicted the Internet Research Americans are likely targets of information efforts by
Agency LLC, located in St. Petersburg, Russia, for foreign adversaries. Additionally, there is a need to
interfering in U.S. elections as far back as 2014 develop evidence-based preventive interventions for
(United States v. Internet Research Agency LLC, 2018). those who are most likely targets within our society.
Given these past and likely extant threats to
U.S. elections, the California Governor’s Office of
Emergency Services asked the RAND Corporation’s Modern Methods Are Rooted in
National Security Research Division for research the Cold War
to help them analyze, forecast, and mitigate threats
We believe that reflexive control theory is, in part,
by foreign actors targeting local, state, and national
the intellectual basis of current information efforts
elections.
targeting the United States that are perpetrated by
This four-part series (Figure 1) will present
Russia and its proxies.4 Scholars have described
• what the literature says about information reflexive control as a means of conveying infor-
efforts by foreign actors mation to others that leads them to make some
• the results of an analysis of social media to predetermined decision (Thomas, 2004). Reflexive
identify potential exploits control theory is a formal theoretical research pro-
• a survey experiment to assess interventions to gram exploring this technique; it was developed by
defend against some of these exploits Vladimir Lefebvre and others and first appeared
• qualitative interviews of survey respondents to in Soviet military literature in the 1960s (Chotikul,
understand their views on falsehoods. 1986; Radin, Demus, and Marcinek, 2020; Thomas,
In this report, we review some of the research 2004). Unlike game theory, this theory does not
on information efforts by foreign actors, focusing assume that individuals act rationally; rather, it
mainly on online environments.2 First, we review assumes that people act “according to their image of
what we believe is the intellectual basis of existing the world and their image of their adversary’s image
Russian information efforts: reflexive control theory.3 of the world.”5 (Appendix A has a more detailed
Second, we review examples of information efforts by description of the reflexive control theory.)
the Soviet Union and Russian Federation. Third, we In general, reflexive control theory presents
the world as a set of binary relationships: People are

FIGURE 1
What This Series Covers

Disinformation Series

PART 1 (this report) PART 2 PART 3 PART 4

Reviews what existing Identifies potential Assesses interventions Explores people’s
research tells us about information exploits in to defend against views on falsehoods
information efforts by social media exploits
foreign actors

2
either in conflict or cooperation with one another,
and their actions are either passive or aggressive in Defining Key Concepts
nature (Lefebvre, 1965, 1966). Although the binary • Information efforts: Activities that “influence,
view is an oversimplification, this assumption is disrupt, corrupt, or usurp the decision-
based on how some historians characterize human making of targets while protesting one’s own”
relationships in the Soviet Union.6 (Chairman of the Joint Chiefs of Staff, 2014,
The model structure implies that a person’s deci- p. A-1).
• Creation and dissemination: Content—
sions depend on what is socially desirable, insofar as
which might be authentic (for example, cre-
they perceive what is desirable to others. Outside par-
ated by human users or trolls) or inauthentic
ties can manipulate this perception, which forms the
(for example, created by bots)—is created and
basis for reflexive control as a tool for understanding
disseminated by state-sponsored actors or
information efforts. Belief in falsehoods is merely their proxies (who could be willing or unwilling
a byproduct of reflexive control, not the end goal. participants).
Disseminating false content is a tactic to manipulate • Information environment: The area in which
one group’s view of others. information efforts exist, broadly defined as
We have identified at least two key features of “the aggregate of individuals, organizations,
reflexive control theory that seem applicable to recent and systems” (Chairman of the Joint Chiefs
information efforts targeting U.S. elections. of Staff, 2014, p. GL-6). These environments
involve any collection of people who interact
with each other online or in person.
One Feature of Reflexive Control
Theory Is to Alter People’s Perceptions
of people find common ground on issues of public
First, the model accepts as a given that relationships concern. After all, it’s difficult to find commonality
between individuals are defined by conflict or collab- on such topics as property taxes or farm subsidies
oration. The interpersonal influences between groups when opponents see each other as un-American or
of people are inputs. Information efforts are trying to racist and each believes these views are deep-seated
shift people’s perceptions, not alter this fundamental within the other.
group structure. For example, reflexive control does This binary framework also suggests a direct
not try to convince people in political party A that way to operationalize reflexive control. Manipulating
they are in conflict with those from political party B. someone’s perception of others is most feasible, and
It assumes this conflict already exists. Instead, these most effective, if that perception can be collapsed
information efforts try to amplify the degree that into a one-dimensional caricature. If someone is “one
people from party A believe that those from party B of them” and if “they” are a monolithic group easily
view them as adversaries. summarized by one characteristic, then perceptions
are more easily manipulated and reinforced. Thus,
we hypothesize that reflexive control in practice
Another Feature Is to Elicit Reactions,
is a priming technique that encourages people to
Not Necessarily to Change Behavior
self-identify with a particular group and to simplify
The second relevant feature of reflexive control the characterization of that group as homogenous
theory is the goal of eliciting reactions from targets. and ultimately in conflict with another group.
The theory views the world as a dichotomy between Thus, foreign adversaries might try to opera-
conflict and collaboration. Successful information tionalize reflexive control by targeting those who are
efforts sow deep divisions between groups of peo- likely to have the strongest reactions to group-based
ple and generate a perception of “us” versus “them” differences. The sources of these differences are
that, in turn, elicits strong reactions in people. The broad: They might focus on race, class, gender,
ultimate goal is to reduce the probability that groups sexual orientation, political affiliation, or geography

3
 on Intelligence of the United States Senate, 2019,
undated). Furthermore, we believe that Russia is an
Active measures are equal opportunity exploiter of social cleavages, as
evidenced by reporting that online Russian trolls
covert and overt appeared to have targeted white supremacists and civil
rights activists (Derysh, 2020; Glaser, 2018; Lockhart,
information efforts 2018; Ward, 2018). In the next section, we discuss some
of the ways that Russia and its proxies appear to con-
organized by a duct these efforts within the United States.
government to affect a
target’s domestic and Russia’s Aim Is to Preserve the
Regime by Weakening the West
foreign politics and are Russia has long used various forms of active
measures—a specific term that falls within the broader
used as an instrument category of information efforts—against the United
of power. States, with a focus on U.S. domestic issues. Active
measures (in Russian, aktivinyye meropriatia or
aktivka) are covert and overt information efforts orga-
(urban versus rural dwellers). Foreign information nized by a government to affect a target’s domestic and
efforts might target these differences by activating foreign politics and are used as an instrument of power
the salient group identity within them and framing (Godson and Shultz, 1985). These measures have
this identity as being in conflict with people in other helped shape the course of international and domestic
groups. By diminishing consensus, adversaries can events in Russia’s favor and helped subvert actions and
potentially cause political paralysis. trends that contradict the government’s intentions.
These tactics and goals are reflected in the Although many countries have sought to use
observations of researchers who have tracked Russian active measures, the Soviet Union and then Russia
information efforts over the past several years. The institutionalized them over many decades and
extensive use of memes focusing on particular groups advanced them into a comprehensive foreign policy
and repeatedly characterizing them in particular tool, particularly against the United States (Allen and
ways works to create a simple but consistent framing Moore, 2018). This tool is used to undermine demo-
of that group. These framings tend to have high emo- cratic governance processes in the United States and
tional resonance and often involve passive statements its allies with the overarching aim of weakening the
about identity (as opposed to calls for action). They United States and advancing Russia as a global power.
prime the viewer to think about “us” versus “them” in This would then support Russia’s view of itself as the
ways that assume the viewer already knows who “we” promoter of a world order and values that are alter-
and “they” are.7 The uses of cross-platform commu- native to the ones represented by the United States
nication and consistent cross-platform branding give and its allies and their view of the liberal rules-based
the impression that these characterizations are more world order (Government of Russia, 2016; Radin and
universal than they actually are.8 Reach, 2017; Stent, 2019).
Russia is not the only foreign actor who con- This section is divided into two parts. First, we
ducts information efforts (see Appendix B for brief discuss the objectives of Russian information efforts in
descriptions of efforts by China, Iran, and Venezuela), the post–Cold War era. Second, we review some of the
but it appears Russia has been an active presence in ways that Russia has applied these efforts against the
the United States in recent years (United States v. United States. The research for this section draws on
Internet Research Agency LLC, 2018; Select Committee open-source publications, such as official documents,

4
research reports and analysis, and commentary and
case study databases by Western and Russian authors. Example 1

An Argument for Every Audience

Russian Information Efforts Reflect During the 2016 presidential election, politically
Four Objectives right-leaning groups were targeted with conspiracy
theories about voter fraud and election stealing. The
In general, a key strategic aim of Russia’s informa- narratives targeting African Americans were largely
tion efforts is to ensure the survival of its regime and apolitical but included attempts at voter suppression
diminish the global dominance of the United States through discouraging them from voting altogether.
and its allies. Russia seeks to make Western-style Politically left-leaning populations were targeted with
democracy less attractive by damaging the global sta- tailored anti-establishment narratives. Lesbian, gay,
tus of the United States, destabilizing it internally, and bisexual, transgender, and other (LGBT+) popula-
weakening its unity with Western allies (Taylor, 2019). tions, Native Americans, and Muslims were targeted
Russia seeks to achieve these aims by using asymmet- with tailored narratives emphasizing the identity and
ric and often long-term methods, which are sowing pride of these groups. (DiResta et al., 2019)
discord, undermining democratic forms of gover-
nance, and seeking to compromise the core values of lead to the creation of a “fifth column”—a group that
the United States (Jenkins, 2019). Moreover, examples is sympathetic to or works in favor of the adversary
of active measures toward Europe suggest that the (Doronin, 2010).
aims of information efforts could be “opaque, as they Open-source reports indicate that Russian-backed
are designed to produce second- and third-order attempts to sow discord in U.S. society have largely
effects” (Pomerantsev, 2015b). In reality, that plays out focused on exploiting racial and immigration issues
as the target continually guesses the real purpose of and that they both make use of existing extremist
the information efforts targeted against it.9 movements across the U.S. political ideological spec-
Open-source reports crystalize Russia’s four main trum and create new such movements. Russian-backed
objectives for its active measures in the United States:10 efforts have even promoted secessionist initiatives
1. Polarize and disrupt societal cohesion by (e.g., CalExit and Texit) (Yates, 2017). Russia and its
exacerbating important and divisive issues, proxies use the strategies of generating extreme anger
such as race, social class, and gender. and suspicion on political issues when trying to co-opt
2. Undermine public confidence in democratic some politically right-leaning groups. When targeting
institutions and processes. some in the African American community, it has tried
3. Spread confusion, generate exhaustion, and to build on the issues of societal alienation and police
create apathy. brutality (Lockhart, 2018).
4. Gain strategic influence over U.S. political
decisionmaking and public opinion. Undermine Public Confidence in Democratic
Institutions and Processes
Polarize and Disrupt Societal Cohesion by The Russian government seeks to undermine the
Exacerbating Divisive Issues U.S. democratic system and institutions by disrupt-
Russia seeks to manipulate important and divisive ing public trust and manipulating the public’s lack
social and political issues as a way to alter public of familiarity with the institutions and processes
opinion, create and exacerbate fissures, and disrupt (Spaulding, Nair, and Nelson, 2019; Thomas, 2015).11
political processes. According to Russian author It tries to achieve this by exploiting the vulnerabilities
Alexander Doronin, subversion by inflating con- of a democratic governance system, especially populist
flict situations, specifically ones that encourage the agendas (Aceves, 2019; Conley et al., 2019; Endicott,
feelings of infringement and dissatisfaction with 2017). Studies on Russian influence in Europe have
those who are perceived to be more privileged, might found that unstable and weak democratic institutions

5
 Example 2 Example 3

Voter Fraud and Suppression Denial and Distraction

During the 2016 elections, Russia-backed actors When accused of shooting down a Malaysia Airlines
appeared to have promoted anti-establishment jet over Ukraine in 2014, Russia’s response was to
sentiments and slandered the U.S. intelligence overwhelm the audience with alternative accusations
community. Several reports observe that digital voter (Pomerantsev, 2014).
suppression campaigns, mostly done via social
networking platforms, specifically targeted African
Americans in an attempt to alienate them from gov- reality, meaning, and facts (Karjukin and Chausov,
ernment institutions and cause them to doubt the 2017). Likewise, Western marketing literature speaks
influence of their votes. (DiResta et al., 2019; Conley about “consumer confusion” through information
et al., 2019; Root and Barclay, 2018) overload and ambiguous and misleading informa-
tion, making consumers less likely to make rational
and political and economic volatility make a coun- choices, less decisive, more anxious, and exhibiting
try more vulnerable to Russian influence, manipu- lower levels of trust and understanding (Walsh and
lation, and even state capture (Conley et al., 2019). Mitchell, 2010). This is in line with RAND’s recent
Disinformation has become a key part of undermining Truth Decay research about the growing conflict
trust between society and government institutions in between fact and opinion, the increasing volume of
democratic countries. Some Western authors draw a opinion over fact, and the public’s diminishing trust
link between the spread of disinformation and declin- in factual information (Kavanagh and Rich, 2018).
ing citizen confidence, which further undermines
trust in official information sources and pushes people Gain Strategic Influence over U.S. Political
toward so-called alternative news sources (Bennett Decisionmaking and Public Opinion
and Livingston, 2018). Russia seeks to gain strategic influence in the U.S.
decisionmaking process by developing access to
Spread Confusion, Generate Exhaustion, and policymaking circles and influential groups and
Generate Apathy increasing control over U.S. public opinion. These
Spreading confusion and obfuscation, specifically targets are assets that might knowingly or unknow-
through disruption or denigration of truthful report- ingly help Russia gain long-term political influence
ing, is one part of Russia’s contemporary propaganda through Russia’s making relatively low-profile
model (Paul and Matthews, 2016). Over time, it investments today. For example, Russian agent Maria
appears that Russia has become more confident in Butina was sentenced to 18 months in prison in
using falsehoods to make people confused, paranoid, 2019 and then deported after the U.S. Department
and passive and to reduce their ability to understand of Justice accused her of “infiltrating organizations
what is true and create a perception that “nothing is having influence in U.S. politics, for the purpose of
ever knowable” (Endicott, 2017; Paul and Matthews, advancing the interests of the Russian Federation”
2016; Soufan Center, 2018). Disseminating intention- (U.S. Department of Justice, 2018; also see Lamond,
ally false news stories online and in traditional media 2018). Prior to that, Russia attempted to use infor-
can give repeated falsehoods legitimacy and has mation efforts to curb fracking in the United States.
the potential to create an environment of confusion The decline in oil prices in 2015 had a severe impact
and disorientation (Bennett and Livingston, 2018). on Russia’s economy, which is highly dependent
Confusion is directly relevant to the ability to make on oil and natural gas. One report concluded that
decisions. One Russian author writes that decision- Russia exploited genuine concerns about the envi-
making is greatly affected by the level of confidence ronmental impact of fracking in its covert efforts to
in signs and their meanings—i.e., confidence about curb the practice, which produces a cheaper source

6
 Russian active measures do still employ
Example 4 more-traditional methods—which in today’s world
Infiltrating Influential Circles are facilitated by the availability of informal com-
munication networks via economic, cultural coop-
In 2018, the Department of Justice charged Maria
eration, expatriate, or proxy individuals or groups,
Butina with being a Russian agent and “infiltrating
as well as other more-conventional means. This
organizations having influence in U.S. politics, for the
seemingly reflects the views of Russian military
purpose of advancing the interests of the Russian
Federation.” The National Rifle Association was one
thinker Colonel S. Leonenko, who pointed out that
of Butina’s targets. (Sullivan, 2019) information technology poses a challenge to reflexive
control because computers lack “the intuitive reason-
ing of a human being” but also offers new methods
of energy that threatens Russia’s exports to Europe.12 of influence—e.g., anonymity, whitewashing of false
As a result, the Department of Justice charged three information, and wide dissemination (Thomas, 2004).
Russian agents with spying on U.S. “efforts to develop Different techniques are not isolated from each other,
alternative energy resources.” Russia’s information and campaigns often move from the virtual and media
campaign included releasing a documentary about environment into the real world by rallying protests
illnesses allegedly triggered by fracking in the United online, hiring individuals to organize rallies, or hiring
States using manipulated testimonies from trusted individuals to participate in rallies and protests.
sources (Rogan, 2015). Russia has also sought to sup- The techniques used might vary because they
port and establish relations with social movements are tailored to the issue, environment, target, and
in the United States and Europe that tend to operate intended result (Doronin, 2010). Russia uses a wide
outside the traditional political party structure, such variety of methods that are tailored to situations,
as secessionist movements in California and Texas.13 and it continually seeks opportunities to achieve
the aims and objectives we have described. Figure 2
Technology Puts New Twists on Cold War
displays several widely used techniques identified in
Techniques
open-source reports.
Although the techniques that were tried and tested
When using these techniques, Russia contacts
during the Cold War have not been forgotten, new
a variety of individuals and groups from across the
technologies and increased cultural and economic
political and social spectrum, often reaching for the
links between Russia and the United States offer
fringe elements on both the political right and left,
a wider array of means and methods by which to
even if Russia is not always ideologically aligned with
manipulate information. Interactions today have
such groups. Its information operation narratives are
been augmented with internet-based media, social
tailored to different target audiences.
networking sites, trolls, and bots. Technological
In Europe, for example, Russia has seduced
development has opened new means of manipulation
different political groups using anti-EU messages,
via information and facilitates more-scalable, bigger,
anti-U.S. hegemony narratives, and appeals to those
and more-effective operations with minimal expense.
interested in preventing fracking (which appears to
The role of the internet has been recognized at the
benefit Russia’s interest in maintaining European
highest levels in Russia. In 2014, Russian President
dependence on Russian gas). In the United States,
Vladimir Putin noted that “the rapid progress of
Russia has seduced some religious conservatives with
electronic media has made news reporting enor-
anti-LGBT+ stances (Pomerantsev, 2014).
mously important and turned it into a formidable
Several U.S. authors have tackled the relation-
weapon that enables public opinion manipulations”
ship between Russia and fringe groups, politically
(Pomerantsev, 2015a). One Russian author further
extreme right- and left-leaning groups, the so-called
elaborates that even small amounts of information
angry young men groups, motorcycle gangs, and
distributed during “crisis situations” might lead to
fight clubs. They note that these groups might
serious results (Doronin, 2010).
become—sometimes unwittingly—Russia’s agents of

7
FIGURE 2
Overview of Select Russian Active Measures

TAILORED DISINFORMATION FALSE ONLINE PERSONAS

Pit different groups against each other by Create false personas, sometimes with
identifying content and topics to which each information that belongs to real people, to hide
targeted group might be most susceptible. real identities.

AMPLIFIED CONSPIRATORY NARRATIVES SOCIAL MEDIA GROUPS

Promote or denigrate an issue, sow distrust, Exacerbate existing issues, gather informa-
and spread confusion by disseminating tion, and recruit for events by creating social
constitutional narratives, rumors, and leaks. media groups dedicated to divisive issues.

PAID ADVERTISING MEMES AND SYMBOLS

Push people to like pages, follow accounts, Utilize memes to create easy-to-share
join events, and visit websites. snippets of information that can emotionally
resonate with people.

AMERICAN ASSET DEVELOPMENT

SECESSIONIST SUPPORT
Reduce the likelihood of detection by
recruiting Americans to perform tasks for Undermine the United States by establishing
handlers. links with and supporting secessionist ideas
and movements.

NARRATIVE LAUNDERING
FRINGE MOVEMENT SUPPORT
Move a narrative from its state-run origins to
the wider media ecosystem through witting or Build support for Russia’s values and society
unwitting participants. by establishing links to extremist groups.

HACK AND LEAK OPERATIONS

Illegally procure information and share via
platforms such as WikiLeaks.

influence. Michael, 2019, writes that Russia’s cultural marriages of convenience or are genuine partnerships
conservatism, nationalist government, and large white based on shared values” (Carpenter, 2018).
population allows it to be perceived as a potential ally In the next section, we review a sample of recent
to the political far right in the United States, especially research to better understand what types of practices
on issues for which some on the political far right feel might help respond to these information efforts.
they are in the minority in a multicultural and liberal
environment. According to Carpenter, 2018, manipu-
lating extreme political groups is part of Russia’s strat- Research Focused on
egy to undermine Western democratic institutions: Responding to Foreign Election
Russia seeks out marginal groups and social outcasts Interference
who can be motivated to fight the institutions of their
In this section, we present results from a systematic
own country and amplify divisive narratives, thus
review of research related to information efforts and
providing Russia with a “shield of deniability” that it
foreign election interference. We reviewed a total of
can use to suggest that any links between Russia and
142 documents and focused on 78 that featured data.
such groups occurred extemporaneously. It is not clear
(For the methodological details of this effort, see
in the literature whether such “partnerships are always
Appendix C.) We begin by describing a framework

8
for organizing this literature, followed by a review of FIGURE 3
some general trends. Framework for Understanding the
Pathology of Falsehoods
Three Features Describe How Russia’s
‘Firehose of Falsehoods’ Works
Russia’s information efforts have been described PRODUCTION DISTRIBUTION CONSUMPTION
as a “firehose of falsehood” because they produce
large volumes of partial truths and objective false-
hoods, continually and repetitively, via multiple
Malinformation
channels—e.g., text, video, audio, imagery (Paul and
Matthews, 2016).14 Here, we describe the pathology of
this firehose. Figure 3 describes three key features of
this pathology: production, distribution, and con- Misinformation
sumption of content (Matthews et al., forthcoming).
We found that the unit of analysis—defined as
the subject (i.e., who or what) of a study—typically
Disinformation
differs for each of the three features in Figure 3.15
For production, the unit of analysis tends to be the
content itself, including malinformation (e.g., leaked
documents), misinformation (e.g., misleading content
that features half-truths), and disinformation (e.g., surrounding these information efforts. For example,
complete falsehoods, such as forgeries or fictitious- some have employed the broad term “hostile social
declarative statements). For distribution, the unit of manipulation,” which encompasses any purposeful,
analysis is the social network on which users share systematic generation and dissemination of harm-
original content or create derivative content to share ful information (Mazarr et al., 2019, p. 15). Recently,
with others.16 For consumption, the unit of analy- scholars have called for a common lexicon for describ-
sis is the individual user who views and shares this ing the characteristics of information efforts (Paul
content—typically, individual users are the general and Matthews, 2018). Given these debates, we use the
public who express their support for content but they framework in Figure 3 as a starting point to help orga-
can also be inauthentic users, such as bots. nize some relevant lines of research.
The research on foreign information efforts does
not always reflect the neatly patterned framework Production Focuses on New Content
described in Figure 3, and there is no shortage of Containing Falsehoods
alternative frameworks and models that describe var- The unit of analysis for research related to produc-
ious features of information efforts. For example, the tion is the content itself. In general, the studies in our
firehose of falsehood framework characterizes Russian sample focused on two topical areas. This first was the
propaganda efforts as producing large volumes of targeting and specific features of content. For exam-
rapid, continuous, and repetitive content through ple, one study examined a data set of 705,381 unique
multiple channels (Paul and Matthews, 2016). Other accounts during the 2016 presidential election to show
research has identified a “Russian disinformation how bots skew perceptions of candidates on social
chain” that describes how Russian leaders use media media (Heredia, Prusa, and Khoshgoftaar, 2018).17 In
proxies (e.g., Russia Today or Sputnik) and amplifi- another study using Twitter data, researchers found
cation channels (e.g., social media platforms or U.S. evidence that Russian information efforts targeted
news media) to reach consumers (Bodine-Baron et al., both conservative and liberal conversations online by
2018). Studies have revisited the terms and definitions impersonating Americans (Starbird, Arif, and Wilson,

9
2018). Although many of these studies focused on U.S. and those focusing on the role of machine-learning
online communities, we did identify relevant studies approaches to accomplish the same task.
from other countries. In one study from China, for First, we found several studies that focused
example, researchers analyzed microblog posts from on the role of social media platforms in this area.
Sina Weibo and Tencent Weibo platforms related to Research on how the conspiracy theory linking
the avian influenza A (H7N9) (Chen et al., 2018). Of vaccines with autism spread via Twitter and Reddit
the 1,680 microblog posts, researchers classified about showed that each platform served different func-
20 percent (n = 341) as misleading. The authors report tions in the dissemination of this disinformation
that users who posted misleading messages had the (Jang et al., 2019). Twitter was found to drive news
highest average rank of reposts compared with other agendas while news content drives Reddit discus-
types of messages, but they ranked lowest in number sions. Understanding how platforms contribute to
of followers and existing posts. different aspects of the diffusion chain would help
The second topical area focuses on automated in detecting the flow of disinformation articles
methods for classifying content that contains and targeting appropriate interventions on these
falsehoods within information environments. For platforms. Likewise, another study investigated
example, one proposed framework for detecting the impact of WhatsApp’s policy that limited the
falsehoods analyzes the text (e.g., title length, number of times a user could forward a message
percentage of proper nouns) and verifies the accu- to just five. On public WhatsApp groups in Brazil,
racy of information compared with a corpus of Indonesia, and India, this policy helped slow
trusted sources (Ibrishimova and Li, 2019). In the spread of disinformation but did not block it
another study, the authors describe a classifier that entirely (de Freitas Melo et al., 2019). The openness
estimates the probability that a news story is false and control of the platform has an impact on the
using such features as the headline (e.g., whether a flow of disinformation. A study comparing Twitter’s
news title had capital letters), authorship character- open platform with Sino Weibo’s more-closed,
istics, sourcing, origin or publisher, and the con- government-controlled platform showed that
tent’s political perspective (Snell et al., 2019). Other sharing misinformation about Ebola outbreaks was
research has applied topic modeling to examine dif- less prevalent on Chinese microblogs (Fung et al.,
ferences in Russian content versus English content 2016). Another study on the spread of the chemtrails
(Chew and Turnley, 2017). conspiracy on several online platforms suggests that
anonymity online appears to help spread conspira-
Distribution Research Explains How cies (Tingley and Wagner, 2017).
Falsehoods Spread Second, there was a growing body of research
The unit of analysis for research related to distribu- about using machine-learning models to track,
tion is the social network. We highlight two types of understand, and mitigate the spread of existing
studies: those focusing on the role of social media plat- falsehoods on social media. One study proposed
forms in preventing the spread of online falsehoods using these models for situational awareness in
understanding a multipolar political landscape
prior to an election. In a study of 60 million
exchanges among 2.4 million Twitter users around
the 2017 French election, researchers qualified
The unit of analysis and quantified various characteristics of online
political communities, tracking their temporal
for research related to evolution, structures, alliances, and semantic
features during the campaign (Gaumont, Panahi,
distribution is the social and Chavalarias, 2018). This situational awareness
might provide a foundation for understanding how
network. falsehoods spread across a country’s electorate

10
and what communities are most affected. Other
studies looked at how to detect disinformation
accurately in these conversations. A graph-based The unit of analysis
machine-learning model tracked the stance of 72
rumors in more than 100,000 tweets in a semisuper- for research related
vised approach (Giasemidis et al., 2020). This model
measured whether the tweeter felt positive, neutral, to consumption is the
or negative about the rumor and was used to help
predict the accuracy of information. Results showed individual consumer.
the algorithm was fast and accurate, exceeding
80 percent average accuracy. Using semisupervised
Additionally, Budak, 2019, researched how the
machine-learning algorithms to classify the stance
2016 presidential election correlated with the preva-
of rumor tweets might lead to fast, scalable, and
lence of false news online. Budak randomly sampled
accurate differentiation of information. Advances in
5,000 tweets each from tweets mentioning President
machine-learning might help researchers accurately
Donald J. Trump or former Secretary of State
classify false information, understand its spread,
Hillary Rodham Clinton from May 2014 to January
and gain insight into how users receive it.
2017. This study showed that the prevalence of false-
hoods in news increased over the course of the cam-
Consumption Research Focuses on the Role
paign and that voter perceptions of former Secretary
of Consumers
of State Clinton were more in line with false news
The unit of analysis for research related to consump- accounts. Finally, in a study of more than 5 million
tion is the individual consumer. Much of the research tweets, researchers applied machine-learning algo-
focused on consumer views of content and the rithms embedded in network models to find user
impacts of fact-checking on these views. attributes that accurately predict how they will react
The first theme focused on consumers’ features to online content (Gallo et al., 2020). Among four
and views surrounding exposure to falsehoods, factors (the Big Five personality traits, time interval,
which could lead to more consumers taking action predominant sentiment, and sentiment distribu-
against false information. Pew surveyed representa- tion), they found that the Big Five personality traits
tive samples of adults in 11 emerging economies and (i.e., openness to experience, conscientiousness,
found that respondents felt they regularly encoun- extroversion, agreeableness, neuroticism)18 had the
tered false information on social media (Smith et al., most impact on user reactions to false information.
2019). Furthermore, an analysis of a three-country A second theme focused on the effects of
survey comparing voters in the United States, the fact-checking of content on consumers, which is a
United Kingdom, and France reports that conserva- popular approach to countering false information.
tive voters in the latter two countries were no more One study looked at Facebook’s “rated false” and
likely than nonconservative voters to report that they “disputed” tags on different headline types (false
were exposed to falsehoods. This was not true in pro-President Trump, false anti-President Trump, and
the United States. Those on the political right in the true headlines) using respondents from Amazon’s
United States reported feeling much more exposed to Mechanical Turk crowdsourcing platform (Clayton et
false news and that they could trust news much less al., 2019). Results showed that both tags reduce beliefs
than did people reporting to be nonconservative vot- in false news, but the rated false tag was more effective
ers (Koc-Michalska et al., 2020). This deterioration than tagging headlines as disputed.
of trust in news media for U.S. conservatives implies Because not all news stories can be checked
that news and government might need to communi- and tagged, providing general warnings might
cate differently with these consumers. help alert users to false news. A related study on
StopFake—a Ukrainian fact-checking organization

11
founded in response to Russian information efforts (1) driving people to view each other as either friends
in 2014—showed that fact-checking organiza- or adversaries, or (2) exhausting people to the point
tions have different focuses. StopFake focused on that they disengage from civic affairs altogether, with
finding falsified evidence, such as manipulated or the result of political paralysis.
misrepresented images and quotes. However, most
U.S. fact-checking organizations assume quotes Russia’s Tactics Aim to Polarize Americans
and images are legitimate and focus on evaluating and Paralyze the U.S. Political Process
nuanced political claims. Some researchers claim These tactics consist of attempts at polarizing and
most U.S. fact-checking groups are ill-equipped disrupting social cohesion. Some tactics aim to
to detect disinformation that is wholly falsified exacerbate divisive issues, such as racial inequities
(Haigh, Haigh, and Kozak, 2018). or immigration. Others target public confidence in
democratic institutions and processes as a way to
undermine social trust. Underlying these efforts is a
Conclusions and broader tactic of using falsehoods to spread confu-
Recommendations sion, drive groups of people to extreme positions, and
This report reviews some of the research that is generate collective exhaustion within U.S. society.
relevant to foreign information efforts targeting U.S. Finally, there is evidence that Russia has tried—and
elections. It provides a general framework for under- continues to try—to gain direct influence over the
standing these efforts and will inform our analysis in U.S. political decisionmaking process, although we
future volumes of this series. We focused on efforts do not know how effective these efforts have been.
by Russia and its proxies because these actors appear
to have been the most active in recent years, but we Our Sample of Relevant Research Revealed
note that other state and nonstate actors also might Some Trends for Responding to Falsehoods
target the United States. As a result of this work, we Although our sample of studies is not represen-
reached four general conclusions. tative of all research on this topic, it does provide
some ideas for emerging practices in responding to
foreign information efforts. Much of this research
Conclusions
is fragmented and cuts across multiple disciplines,
Foreign Interference in U.S. Politics Is Not a causing us to organize it by primary unit of analysis:
New Phenomenon the production of new falsehoods, the distribution of
Foreign influence in U.S. domestic affairs dates back existing falsehoods, or the consumers of this content.
to the founding of this country, and there are several Research on production largely focused on tar-
examples in our 244 years of existence. geting of falsehoods and the features of this content.
For studies on the distribution of existing falsehoods,
How the Russians Have Tried to Interfere in research focused on the role of social media plat-
Recent U.S. Elections Follows Some Logic forms in preventing the spread of online falsehoods
and the role of machine-learning models to miti-
We hypothesize that reflexive control theory—a
gate this spread. Finally, research on consumption
theoretical research program first developed in the
largely focused on consumer views of content and the
1960s and used by the Soviet military—is part of the
impacts of fact-checking.
intellectual basis for current Russian efforts. At its
core, reflexive control theory assumes that people live
in a polarized world defined by either cooperation Recommendations for Responding to
or conflict and that people make decisions based on Foreign Information Efforts
these views. We believe that Russia is trying to gener- Foreign interference has occurred throughout U.S.
ate, spread, and amplify falsehoods that distort views history and likely will continue in the future. Russia
of “us” versus “them,” with the desired outcomes of

12
seems to have advanced its information efforts in that hold meaning for people are useful starting
recent years, and we suspect other countries will try points because false content tied to these character-
to emulate these practices. We offer three recom- istics might elicit strong reactions (Marwick, 2018).
mendations for how to start designing responses to We suspect that foreign efforts will likely produce
these existing and emerging threats that target U.S. content that plays on these identities in an effort to
democracy. In future volumes of this series, we will amplify differences and deepen preexisting fault lines
present results with more-specific recommendations in U.S. society. Thus, we recommend developing
for responding to these foreign information efforts. strategies that anticipate which subgroups are most
vulnerable to such efforts without publicly shaming
A Holistic Strategy Is the Optimal Response these groups or targeting specific individuals.
to Information Efforts by Foreign Countries
During the Cold War, Secretary of State Lawrence Any Response Should Attempt to Protect
Eagleburger recommended a “balanced approach” Potential Targets Against Foreign Information
to Soviet information efforts that neither ignores the Efforts
threat nor becomes obsessed with it (Eagleburger, The antidote to manufacturing intergroup conflict is
1983). Our assumption is that reflexive control theory convincing people that they have more in common
is part of the intellectual basis for Russian efforts tar- with those who are different from them than they
geting U.S. elections. The unit of analysis of this the- might believe at first glance. We recommend collect-
ory is broad, spanning the entirety of U.S. society and ing, analyzing, and evaluating preventative interven-
any particular piece of online content, social media tions to protect people from reacting to falsehoods
platform, or individual consumer. We recommend meant to divide the country (e.g., public campaigns
that any defensive strategy account for the complex that emphasize shared interests of Californians,
relationships among the production of falsehoods, how public warnings about broader information efforts by
others distribute content (particularly online), and the foreign adversaries, or media literacy programs for
impacts of this content on consumers. subgroups that are potential targets).
In conclusion, democracy depends on citizens
Any Defense Should Anticipate Those Who finding consensus with people whom they might view
Are Likely to Become Targets of These Efforts as different from them. Foreign adversaries have made
We believe that a key goal for information efforts is attempts at undermining the formation of this con-
to alter people’s perceptions to amplify a view of “us sensus and will continue to do so. There is a logic to
versus them,” with political paralysis as the ultimate these attempts. The best defense is a holistic approach
goal. Social or political issues tied to identities (such that accounts for the preexisting fault lines that exist
as race, gender, social class, or political affiliation) within U.S. society.

Democracy depends on citizens finding

consensus with people whom they might view
as different from them. Foreign adversaries have
made attempts at undermining the formation of
this consensus and will continue to do so.
13
Appendix A: Detailed “aggressive” behaviors—or, as a more specific exam-
Description of Reflexive Control ple, violence versus peacebuilding. Given the assumed
ethical norms and the types of relationships that bind
Theory
people on the network, actors might behave differently
The mathematical models of reflexive control theory depending on the influence exerted by the others.
evolved from a scenario with one decisionmaker torn For example, under some conditions, everyone
between two actions to a group of decisionmakers in a network might decide to adopt the “aggressive”
embedded within a social network, each of whom behavior. With a different combination of influences,
must choose one of two actions (Lefebvre, 1966, 1980, some people might switch their behavior. In yet other
2009, 2010). These decisionmakers operate in a world situations, no behavior might be consistent with a per-
of binary relationships. First, the theory assumes that son’s relationships and influences; the actors are then
others in the network are either friends or enemies of considered “frustrated” and cannot make a choice.21
the actor. Second, reflexive control theory assumes Each person’s chosen behavior is the solution to a
that the ethical belief system of the society in which mathematical formula—drawing from graph theory
actors exist is also binary: Either the ends justify the and abstract algebra—that captures the reflexive
means or they do not. Finally, the theory assigns each concept at the core of this theory.22 Specifically,
person’s influence a value of either 0 or a 1 (in less people’s decisions depend not only on their views of
neutral terms, influence can be thought of as urg- themselves and others (as described by the network
ing someone to choose between two polar opposites of relationships in which they exist), but also on how
on a spectrum of behaviors).19 Finally, according to others view them and everyone else as described by
reflexive control theory, one person might influence the network of relationships. For example, let us say
another in these direct and indirect ways depending that two people named A and B are friends. A thinks
on the parameters over which they exert control.20 of herself as being a friend of B and views B as being
Furthermore, the outcome is also binary: Each her friend. She also believes that B thinks of A as a
person might choose between two polar opposite person who is her friend. The formulae determining
actions—represented by a value of 0 or 1—while each person’s action captures this series of recursive
being influenced by their neighbors in the social self-images: how you view yourself, how you view
network. The choices can be thought of as “passive” or others, and how you think others view you.

14
Appendix B: Overview of China, politicians (Mattis, 2018; Mazarr et al., 2019; Watts,
Iran, and Venezuela’s Influence 2020). China is primarily interested in achieving
longer-term policy objectives (Doshi and Williams,
Efforts Targeting the United
2018). One Chinese author explains that public
States opinion warfare “during peacetime pays more
Although Russia appears to be the most active and attention to long-term infiltration into the objects
persistent state actor attempting to influence U.S. of the society’s and culture’s deep structure, chang-
domestic politics, other countries have also pursued ing the awareness and conviction of the enemy
foreign policy goals by seeking to influence U.S. masses” (Wang Lin and Wang Guibin, 2004, cited
decisionmakers and society via online news and in in Beauchamp-Mustafaga and Chase, 2019).
social media. This section discusses how China, Iran, Until recently, China’s external influence efforts in
and Venezuela approach influence operations.23 The both traditional and new media mostly focused on
Director of National Intelligence and major social burnishing China’s image and achieving favorable
media sites (e.g., Facebook and Twitter) have iden- reactions.
tified these countries as having carried out online Lately, however, China has been more aggressive
information campaigns that target the United States, in using online media to spread divisions in target
and their previous activities suggest they might try groups, echoing the online techniques used by Russia
again in 2020.24 (Watts, 2020). Figure B.1 displays an overview of
select Chinese-government narratives targeting the
United States. China has allegedly sought to influ-
China
ence the population, domestic politics, and election
China considers itself to be in an ideological com- processes of the United States, although not on
petition with the West and the United States, and the same scale as Russia. For example, China has
the internet to be its battlefield. Although China has been accused of trying to make illegal donations
a long history of using information manipulation during the 1996 U.S. presidential campaign and to
techniques, censorship, and propaganda to achieve gubernatorial and congressional campaigns in 2013
domestic or regional effects, campaigns beyond its and 2018, and the placement of paid propaganda
immediate neighborhood have lacked the range and newspaper inserts has been part of China’s toolbox
finesse of Russian foreign influence operations.25 Its for some time (Beauchamp-Mustafaga and Chase,
influence activities are embedded into its regional 2019). For example, in September 2018, leading up
and global policy objectives, which consist of pre- to elections, the Chinese government published a
venting the United States from curtailing China’s paid insert in the Des Moines Register highlight-
global influence and destroying U.S. global and ing Chinese President Xi Jinping’s ties to Iowa, the
regional influence.26 According to Diamond and benefits to Iowans of trade with China, and the costs
Schell, 2018, China is also at least partly motivated of President Trump’s trade war (Niquette and Jacobs,
by the view that U.S. ideals and values of freedom are 2018). Although there is limited evidence of Chinese
a “direct challenge to its defense of its own form of interference with the 2016 presidential elections,
one-party rule.”27 one report observes that the spreading social media
Chinese influence operations outside the imme- owned by Chinese companies (for example, WeChat)
diate region use overt and covert measures. They might provide China with more tools of influence.
tend to focus on establishing and exploiting human Although Microsoft claimed in 2019 that China’s
relationships, specifically targeting ethnic Chinese. hacking threat to political groups is not as signif-
These operations try to cultivate relationships with icant as those posed by Russia and Iran, Google
key foreign decisionmakers, opinion leaders, and announced more recently that Chinese hackers
the business community; inject Chinese narratives are targeting the personal accounts of former Vice
into foreign educational establishments, media, President Jospeh Biden (Sanger and Perlroth, 2020).
and public opinion; and tarnish the reputation of China-linked Twitter accounts have also been found

15
FIGURE B.1
Overview of Key Selected Chinese Government Narratives

BENIGN NONPOLITICAL BENIGN POLITICAL PERSUASIVE HOSTILE

NARRATIVES NARRATIVES NARRATIVES NARRATIVES
• China’s beautiful • Visits of Chinese • Touting the benefits • Western political
landscapes and leaders of free trade for U.S. interference in
scenery • Explanation of events farmers Chinese domestic
• China’s heritage via the Party lens • Discussing the risks issues (e.g., Hong
• China’s • Touting the Party and of U.S.-Chinese Kong protests,
accomplishments China’s leadership economic tensions Taiwan)
and positive impact
on the world

to recycle tweets calling for the independence of text (Wallis et al., 2020). China’s use of disinforma-
California and comparing the “Yes California” cam- tion techniques during the 2019 coronavirus pan-
paign with the Hong Kong independence movement. demic suggests a willingness to transition from a
However, it is likely that the aim of this activity was strategy of avoiding confrontation and controversial
to liken the independence of Taiwan to California issues to embracing more-aggressive tactics, such as
breaking away from the rest of the United States and promoting conspiracy theories and sowing confusion
trying to discredit protestors in Hong Kong (Thomas (Kendall-Taylor and Shullman, 2020; King, Pan, and
and Zhang, 2020). Roberts, 2017). Although Chinese writers discuss
The populism and divisions in the United States the value of tailored messaging to different audi-
today might present China with an opportunity to ences, there is little evidence so far of such nuanced
promote the values supported by the Chinese gov- narratives.
ernment and carry out targeted messaging.28 Experts China could also seek to establish a network
suggest that China is increasingly seeking to cultivate among younger population groups. One Chinese
influence abroad via Western-based online platforms, author, when discussing propaganda in Hong Kong,
as illustrated by Chinese government organizations writes that China should “use social networking plat-
and government-supported media agencies open- forms such as Facebook and Twitter to establish vir-
ing accounts on Western-based social media sites tual, voluntary, and loose groups amongst all social
(Bradshaw and Howard, 2019). Experts expect China groups, especially the youth, [so that] at any time and
to seek to create confusion in the U.S. information any place we can push to them web postings we have
space by making use of existing conflicts, such as processed and edited to propagandize our ideas” (Yu,
exacerbating public scandals, disseminating rumors, cited in Mazarr et al., 2019).
or publishing personal information to denigrate
political or community leaders.29
Iran
China seems to have little interest in creating
complex false online personas (Wallis et al., 2020). Iran’s online influence techniques are becoming
Analysis of China’s social media activities show that increasingly sophisticated and diverse (Hanlon,
“astroturfing” and other covert means of dissemi- 2018). Figure B.2 displays an overview of select aims
nating pro-China narratives are among its preferred and techniques of Iran’s influence efforts targeting
online methods, specifically when accomplished via the United States. Although Iran’s online operations
hirelings or purchased bots that offer plausible deni- were considered clumsy just a few years ago, its
ability.30 One analysis of Twitter data also suggests a methods have become more subtle and varied. Recent
penchant for images embedded in Chinese-language analysis indicates that Iran’s techniques now consist

16
FIGURE B.2
Overview of Select Aims and Techniques of the Iranian Government’s Influence Efforts
Targeting the United States

MAIN AIMS MAIN TECHNIQUES

• Disseminate Pro-Iranian narratives • Iranian TV, radio networks in English

• Exaggerate Iran’s moral authority • Social media posts in English by Iranian leaders,
• Polarize U.S. society state-supported media networks
• Discredit U.S. critiques of Iran • Doctored photos and memes
• False online personas (designed to appear American)
• Botnets and online trolls

of information operations, social engineering via International Studies, undated; Martin and Shapiro,
LinkedIn, denial-of-service attacks, defacement, and 2019). Analysis by the data analysis firm Graphika,
hacking techniques (Glyer and Carr, 2020). Iranian 2020, shows that Iran is mainly focused on pursuing
manufactured online personas have evolved from the foreign policy goals of discrediting the U.S. cri-
having an obvious affiliation with Iran or inauthen- tique of human rights in Iran and the U.S. sanctions
tic news websites supported by the Iranian govern- against Iran.
ment (e.g., Liberty Front Press) into more-authentic Analysts note that Iranian online influence
American personas or impersonations of real operations against the United States so far have
Americans who appear to be affiliated with local pro- sought to polarize the U.S. population and politics
gressive movements (Revelli and Foster, 2019). Iran by distributing divisive content on social media and
has also placed propaganda in genuine online and amplifying conspiracy theories (e.g., related to the
print media outlets in the United States in an effort terrorist attacks of September 11, 2001), disseminat-
to promote Iranian interests (FireEye Intelligence, ing pro-Iranian propaganda, and discrediting U.S.
2018; Nimmo, 2018). For example, Facebook in critique of Iran’s policies and human rights record
2020 removed an online network of hundreds of (Martin and Shapiro, 2019). There is no evidence
fake social media accounts that had been spreading of substantial Iranian attempts to influence U.S.
Iranian propaganda since 2011. A 2018 investigation elections, but Iran’s efforts to support Scotland’s
by the Reuters news agency uncovered more than referendum to achieve independence from the United
70 inauthentic news websites disseminating pro- Kingdom in 2014 indicate that Iran has experimented
paganda in more than 15 countries, and Microsoft with online election meddling beyond its immediate
revealed in 2019 that it had detected significant neighborhood (Stubbs and Paul, 2020; Bradshaw and
malign cyber activity from Iran (Burt, 2019). Howard, 2019).
Iran uses cyber operations—a relatively cheap
means to gain influence while ensuring plausible
Venezuela
deniability—as foreign policy and statecraft. The
Iranian government has been accused of carrying out Venezuela has a history of trying to pursue its
several cyberattacks against U.S. political organiza- anti-U.S. foreign policy in the United States through
tions,31 and Iran or its proxies have allegedly sought social power—providing generous financial handouts
to influence or attack U.S. elections and political and developing networks with socially disadvantaged
leaders. During the 2016 U.S. presidential elections, organizations or other groups. The aim is to gener-
Iranian trolls posted anti-President Trump messages, ate support for the Venezuelan government and its
and Iran has also attacked U.S. government officials ideological stance and to use these efforts as publicity
and Iranians living abroad (Center for Strategic and stunts (Corrales, 2011). The cases discussed in the

17
literature show Venezuela’s preference for the use of complex information environment that has multiple
proxies, such as nongovernmental organizations or actors and narratives, and the true motives or origin
business enterprises in the United States, that at least of the disseminated information might be obscured
superficially shield Venezuela from the effort.32 from the consumer. Iran, Venezuela, and particularly
Venezuela seeks to pursue several aims: positively China seem to be increasingly invested in developing
portray its government, criticize the U.S govern- online influence techniques, which might further
ment, and sow divisions within the United States complicate efforts to detect, identify, and respond
(Tromblay, 2018). Recent Twitter data analysis also to them. U.S. adversaries seem specifically inter-
shows that Venezuela and its proxies have started ested in online influence operations because they
using online news and social media to pursue these are cost-effective and provide plausible deniability.
goals: In 2019, Twitter removed accounts located in Increased societal polarization and discord further
Venezuela that were “engaging in platform manipula- fertilize the U.S. ground for foreign influence opera-
tion targeted outside of the country” (Roth, 2019). tions, allowing foreign actors to exploit the interests
and disagreements among different social, political,
or other camps. Furthermore, existing analysis notes
Conclusion
that small groups and weaker actors might achieve a
Online influence operations are often difficult to disproportionally large effect by disseminating false
detect and categorize. They are carried out in a information online.

18
Appendix C: Systematic Using a subsample of 40 documents, our team of
Literature Review analysts were in agreement on which documents to
use in the final sample 85.8 percent of the time.34
We developed a search string that identified docu- Our sample of documents does not represent
ments related to the creation, spread, and consump- all research on responding to information efforts.
tion of falsehoods on a variety of media, such as The topic cuts across many areas of research, such
the internet, television, and radio. This string also as communications studies, sociology, psychology,
covered mitigation strategies and four key state-based political science, public policy, economics, computer
foreign actors: Russia, China, Iran, and Venezuela. science, and philosophy. Thus, our sample of articles,
We ran versions of this search string on 13 academic book chapters, and reports provides a snapshot to
databases and the Defense Technical Information better understand the types of topics that are studied.
Center database.33 We focused on studies published Toward this end, we developed the following search
from 2013 to April 2020. The searches yielded string to identify relevant studies in our sample:
317 documents; we determined that 44.8 percent
(142 documents) of them were relevant to this study. (TITLE-ABS(misinformation OR disinforma-
tion OR false OR mislead OR “fake news”)
Figure C.1 displays the documents we used or dis-
carded in this systematic literature review. AND TITLE-ABS(internet or “social media”
Next, four analysts split up 142 documents in OR news OR television OR radio OR *phone*
our final sample for review. We used a three-step OR email OR mail)
approach for making this decision:
1. The complete document must have been writ- AND TITLE-ABS(spread OR propogat* OR
ten in English. disseminat* OR circulat* OR communicat* OR
diffuse OR broadcast)
2. It must relate to information efforts.
3. The documents must use some type of quali- AND TITLE-ABS(Mitigat* OR Interven* or
tative or quantitative data. Counter* or Measure* or reduc* or increas* or
decreas*)

FIGURE C.1 And TITLE-ABS(elect* or Russia* OR China

OR Chinese OR Iran* OR Venezuela* OR
Breakdown of Documents Considered in
domestic)
this Analysis
AND pubyear aft 2013)
Total sample of documents
n = 317
We ran this search string in the following databases:
• Academic Search Complete
• APA Psycinfo
• Applied Science and Technology Full Text
Title and
Title and abstract • Business Source Complete
abstract outside
within
• CINAHL Plus with Full Text ( Cumulative
of scope
scope n = 175 (55.2%) Index to Nursing and Allied Health
n = 142 (44.8%)
Literature)
• Index to Legal Periodicals & Books
Contents within • Library, Information Science, and Technology
scope, but do Abstracts (LISTA)
not have data
n = 64 (45.1%)
• Military Database
Contents within
scope and have data
• PAIS Index (Public Affairs Information
n = 78 (54.9%) Service)

19
 • PolicyFile AND (Citation(spread OR propogat* OR
• Scopus disseminat* OR circulat* OR  communicat*
• Sociological Abstracts OR diffuse OR broadcast)
• Web of Science.
AND (Citation(mitigat* OR interven* OR
We then ran the following similar but modified counter* OR measure* OR reduc* OR increas*
string in the Defense Technical Information Center OR decreas*)
database, limited to accessible reports only:35
AND (Citation(election* OR russia* OR china
Citation(misinformation OR disinformation
OR chinese OR iran* OR venezuela* OR
OR false OR mislead OR “fake news”)
domestic)
AND (Citation(internet OR “social media” OR
AND (Citation(“information operations” OR
news OR television OR radio OR *phone* OR
“info ops” OR “psychological operations”
email OR mail)
OR psyops OR “psy-ops” OR “information
warfare”)

20
 11
Notes For a more detailed discussion of Russia’s approach to infor-
mation warfare, see Thomas, 2004.
1
We note that the lines between foreign and domestic are 12
blurred surrounding information efforts. Rogan, 2015, concludes that Russia used a “three-pronged
strategy” and tried to (1) make covert payments to well-
2
This review provides a framework for interpreting the results intentioned environmental groups in the West, often even with-
in subsequent parts of this series. We draw from the Chairman out their knowledge, (2) gather intelligence on the U.S. energy
of the Joint Chiefs of Staff in defining what information efforts industry, and (3) mount an information campaign against
are and where they take place, and define information efforts as fracking, calling it a hoax.
activities that “influence, disrupt, corrupt, or usurp the decision 13
making of targets while protesting one’s own” (Chairman Tweets originating in Russia supported California and Texas
of the Joint Chiefs of Staff, 2014, p. A-1). These efforts might secession movements (Martin and Shapiro, 2019).
include authentic content (such as that from human trolls) and 14
We focus on information efforts by Russia and their proxies
inauthentic content (such as that from bots) that is created and because they appear to be some of the most organized in the
disseminated by state-sponsored actors or their proxies (who world. Furthermore, much of the contemporary research on
could be willing or unwilling participants). Such efforts exist in this topic is focused on Russia and its proxies. We acknowledge,
an information environment, broadly defined as “the aggregate however, that other countries also engage in similar efforts. For
of individuals, organizations, and systems” (Chairman of the more information on those efforts, see Appendix B.
Joint Chiefs of Staff, 2014, p. GL-6). These environments involve
15
any collection of people who interact with each other online or Research on information efforts often touches on different
in person. (For more details, see Chairman of the Joint Chiefs of units of analysis. Thus, many documents in our sample are clas-
Staff, 2014.) sified across the pathology of falsehoods described in Figure 3.
See Lewis-Beck, Bryman, and Liao, 2004.
3
Vladimir Lefebvre personal e-mail with the lead author,
16
March 3, 2018. We broadly define users to include humans and software
agents, commonly known as bots.
4
This hypothesis is largely based on past research, but Vladimir
17
Lefebvre confirmed that while he did not have specific infor- This research could be categorized under consumption
mation concerning Russia’s use of the theory, it was his belief because it also looked at public opinion as measured by senti-
that the Russian Federation uses reflexive control theory in its ment and volume baselines in the data.
information efforts against the United States (Lefebvre personal 18
The OCEAN model/Big-Five personality traits in this article
e-mail with the lead author, March 3, 2018). assigned scores to people’s character based on the following
5
The mathematical framework describing reflexive control is five dimensions: openness, conscientiousness, extraversion,
distinct from the language of most game theory, although both agreeableness, and neuroticism. For more details, see Gallo et al.,
deal with the strategic use of information. Reflexive control 2020, p. 921.
theory does not use an equilibrium concept to predict outcomes, 19
Formally, the social network is a complete graph whose edges
and it does not assume agents behave rationally according to a comprise two disjoint subsets separating contentious from coop-
utility function; rather, it assumes agent behavior is internally erative relationships. Every such graph with N that meets a sim-
consistent with an ethical belief function (Lefebvre and Farley, ple criterion can be mapped to a Boolean function in N variables,
2007, p. 636). where the operations + and x correspond to either contentious or
6
In general, reflexive control theory assumes that ethical frame- cooperative edges. The action chosen by person n is calculated by
works differ in the United States and Soviet Union: The former setting n equal to the function, inputting others’ influences on n
assumes that bad ethical means should not be implemented for for the other variables, and solving for n. There are extensions of
good ethical outcomes; the latter is less concerned when there is the model that incorporate more than two possible actions, but
conflict between bad means and good goals. For more details, see these extensions are somewhat stylized and do not always have
Chotikul, 1986; Lefebvre and Farley, 2007; and Umpleby, 2016. intuitive real-world interpretations.
20
7
See, for example, the “roster of identities” listed by DiResta Chapter 9 of Lefebvre, 2010, refers to these as “conscious” and
et al., 2019, p. 11. “subconscious” influences.
21
8
The cross-platform tactic has been described as a “media For a benign example of a son frustrated by his parents’ influ-
mirage” that creates an “immersive information ecosystem,” ences in choosing a spouse, see Chapter 10 of Lefebvre, 2010.
(DiResta et al., 2019, pp. 14, 42; Howard et al., 2018, p. 8). 22
In some cases, the solution might be a fraction between 0 and
9
Pomerantsev, 2015b, provides an example: Estonia must con- 1, which would then be interpreted as the probability the person
stantly guess whether the Kremlin’s threats about Russia having chooses the action labeled 1.
the capabilities to invade Estonia are an effort to show real 23
This overview does not present a comprehensive discussion
intention, affect the morale of the Estonian population, or seek of Chinese, Iranian, and Venezuelan online influence efforts.
publicity from reporters elsewhere. Rather, it considers key online-based influence elements that
10
Note that this selection of key objectives is based on an analysis these countries have used or might use in the future to affect U.S.
of a selection of open-source publications and is focused on aims domestic politics.
that are relevant for U.S. domestic politics only. 24
Facebook and Twitter have removed accounts engaged in
“coordinated inauthentic behavior” that is linked to a gov-

21
ernment actor or a political party and are engaged in foreign References
influence operations. Note that this discussion only discusses
foreign-focused influence efforts from these countries, with a Aceves, William J., “Virtual Hatred: How Russia Tried to Start a
specific focus on the United States. Our analysis is not intended Race War in the United States,” Michigan Journal of Race & Law,
as a comprehensive account of the strategies, means, and meth- Vol. 24, No. 177, 2019.
ods of these countries or the information threats that the United Allen, T. S., and A. J. Moore, “Victory Without Casualties:
States might encounter, and is based only on relevant literature Russia’s Information Operations,” Parameters, Vol. 48, No. 1,
that has already been published. (Bradshaw and Howard, 2019; Spring 2018, pp. 59–71.
Sebenius, 2019) “Astroturfing,” Merriam-Webster.com, undated. As of July 29,
25
For example, China’s external influence activities in the 2020:
region have specifically focused on Taiwan. https://www.merriam-webster.com/dictionary/astroturfing

26 Beauchamp-Mustafaga, Nathan, and Michael S. Chase,

For a discussion of China’s foreign policy and influence activ-
Borrowing a Boat Out to Sea: The Chinese Military’s Use of
ities, see Mazarr et al., 2019. Social Media for Influence Operations, John Hopkins School of
27
For an in-depth discussion on China’s approach to influence Advanced International Studies, policy paper, 2019. As of July 29,
2020:
operations, see Beauchamp-Mustafaga and Chase, 2019.
https://www.fpi.sais-jhu.edu/borrowing-a-boat-out-to-sea-pdf
28
Likewise, China could be expected to tie in existing or emerg- Bennett, W. Lance, and Steven Livingston, “The Disinformation
ing issues in the United States with China’s overall foreign policy Order: Disruptive Communication and the Decline of
objectives. One such example would be using racial discrimi- Democratic Institutions,” European Journal of Communications,
nation issues to hurt the reputation of the United States with its Vol. 33, No. 2, April 2, 2018, pp. 122–139. As of July 20, 2020:
allies or to distract from China’s abysmal human rights record. https://journals.sagepub.com/doi/full/10.1177/0267323118760317
(Mazarr et al., 2019; Wallis et al., 2020)
Bodine-Baron, Elizabeth, Todd C. Helmus, Andrew Radin, and
29
This assertion is based on an analysis of Chinese authors in Elina Treyger, Countering Russian Social Media Influence, Santa
Beauchamp and Chase, 2019. Monica, Calif.: RAND Corporation, RR-2740-RC, 2018. As of
July 20, 2020:
30
Astroturfing refers to organized activities that intend to create https://www.rand.org/pubs/research_reports/RR2740.html
a false impression of widespread support for a position (“Astro-
Bradshaw, Samantha, and Philip N. Howard, The Global
turfing,” undated). Disinformation Order: 2019 Global Inventory of Organised Social
31
A report by Microsoft names Iran among the countries that Media Manipulation, Oxford, England: The Computational
Propaganda Project at the Oxford Internet Institute, 2019.
carried out hacking attempts in 2019 (Volz, 2019).
32
Budak, Ceren, “What Happened? The Spread of Fake News
The Bolivarian Circles, for example, was created in 2001 and Publisher Content During the 2016 U.S. Presidential Election,”
is a group that seeks to promote the values of Simon Bolivar but WWW ‘19: The World Wide Web Conference, May 13–17, 2019,
was also a means for neighborhood groups to request assistance San Francisco, CA, USA, Geneva, Switzerland: International
for community programs directly from the President of Venezu- World Wide Web Conference Committee, May 2019, pp. 139–
ela Hugo Chavez (U.S. Bureau of Citizenship and Immigration 150. As of August 1, 2020:
Services, Resource Information Center, 2002). https://dl.acm.org/doi/pdf/10.1145/3308558.3313721
33
Details of these search strings and the databases are provided Burt, Tom, “New Cyberthreats Require New Ways to Protect
in Appendix C. Democracy,” Microsoft On the Issues,” blog post, July 17, 2019.
As of September 1, 2020:
34
We estimated five different agreement coefficients among the https://blogs.microsoft.com/on-the-issues/2019/07/17/new-
four raters for a randomly selected subset of 40 documents that cyberthreats-require-new-ways-to-protect-democracy/
were coded by all raters. Each of these coefficients were above Carpenter, Michael, “Russia Is Co-Opting Angry Young Men,”
widely accepted benchmarks for interrater reliability, including The Atlantic, August 29, 2018. As of August 31, 2020:
Gwet’s Agreement Coefficient (0.69) and Krippendorf’s Alpha https://www.theatlantic.com/ideas/archive/2018/08/
(0.69). (For more details, see Klein, 2018, p. 880.) russia-is-co-opting-angry-young-men/568741/
35
We modified this search term in DTIC because an initial Center for Strategic and International Studies, “Significant Cyber
screening with the original parameters led to an overwhelm- Incidents,” webpage, undated. As of June 5, 2020:
ing amount of false positives related to electronic warfare and https://www.csis.org/programs/technology-policy-program/
command, control, communications, computers, intelligence, significant-cyber-incidents
surveillance and reconnaissance (C4ISR) in military operations. Central Intelligence Agency, Intelligence in the Civil War,
Washington, D.C., undated. As of July 29, 2020:
https://www.cia.gov/library/publications/intelligence-history/
civil-war/Intel_in_the_CW1.pdf
Chairman of the Joint Chiefs of Staff, Joint Information
Operations Proponent, Instruction 3210.01C, February 14, 2014.
As of July 29, 2020:
https://www.jcs.mil/Portals/36/Documents/Library/
Instructions/3210_01.pdf

22
Chen, Bin, Jian Shao, Kui Liu, Gaofeng Cai, Zhenggang Jiang, Doshi, Rush, and Robert D. Williams, “Is China Interfering in
Yuru Huang, Hua Gu, and Jianmin Jiang, “Does Eating Chicken American Politics?” Lawfare, October 1, 2018. As of September 1,
Feet with Pickled Peppers Cause Avian Influenza? Observational 2020:
Case Study on Chinese Social Media During the Avian Influenza https://www.lawfareblog.com/china-interfering-american-
A (H7N9) Outbreak,” JMIR Public Health and Surveillance, politics
Vol. 4, No. 1, January–March 2018.
Eagleburger, Lawrence S., “Unacceptable Intervention: Soviet
Chew, Peter A., and Jessica G. Turnley, “Understanding Russian Active Measures,” NATO Review, Vol. 31, No. 1, 1983.
Information Operations Using Unsupervised Multilingual Topic
Modeling,” in Dongwon Lee, Yu-Ru Lin, Nathaniel Osgood, and Endicott, Marisa, “Propaganda’s New Goals: Create Confusion,
Robert Thomson, eds., Social, Cultural, and Behavioral Modeling: Sow Doubt,” U.S. News, January 31, 2017. As of July 20, 2020:
10th International Conference on Social Computing, Behavioral- https://www.usnews.com/news/national-news/
Cultural Modeling and Prediction and Behavior Representation articles/2017-01-31/russian-propagandas-new-goals-create-
in Modeling and Simulation, London: Springer Nature, 2017, confusion-sow-doubt
pp. 102–107. FireEye Intelligence, “Suspected Iranian Influence Operation
Chotikul, Diane, The Soviet Theory of Reflexive Control in Leverages Network of Inauthentic News Sites & Social Media
Historical and Psychocultural Perspective: A Preliminary Study, Targeting Audiences in U.S., UK, Latin America, Middle East,”
Monterey, Calif.: Naval Postgraduate School, No. NPS55-86-013, blog post, August 21, 2018. As of September 1, 2020:
1986. https://www.fireeye.com/blog/threat-research/2018/08/
suspected-iranian-influence-operation.html
Clayton, Katherine, Spencer Blair, Jonathan A. Busam, Samuel
Forstner, John Glance, Guy Green, Anna Kawata, Akhila Fung, Isaac Chun-Hai, King-Wa Fu, Chung-Hong Chan,
Kovvuri, Jonathan Martin, Evan Morgan, et al., “Real Solutions Benedict Shing Bun Chan, Chi-Ngai Cheung, Thomas Abraham,
for Fake News? Measuring the Effectiveness of General Warnings and Zion Tsz Ho Tse, “Social Media’s Initial Reaction to
and Fact-Check Tags in Reducing Belief in False Stories on Social Information and Misinformation on Ebola, August 2014: Facts
Media,” Political Behavior, February 2019, pp. 1–23. and Rumors,” Public Health Reports, Vol. 131, No. 3, May–June
2016, pp. 461–473.
Conley, Heather A., Donatienne Ruy, Ruslan Stefanov, and
Martin Vladimirov, The Kremlin Playbook 2: The Enablers, Gallo, Fabio R., Gerardo I. Simari, Maria Vanina Martinez,
Washington, D.C.: Center for Strategic and International Studies, and Marcelo A. Falappa, “Predicting User Reactions to Twitter
March 2019. Feed Content Based on Personality Type and Social Cues,”
Future Generation Computer Systems, Vol. 110, September 2020,
Corrales, Javier, “Conflicting Goals in Venezuela’s Foreign pp. 918–930.
Policy,” in Ralph S. Clem and Anthony P. Maingot, eds.,
Venezuela’s Petro-Diplomacy: Hugo Chávez’s Foreign Policy, Gaumont, Noé, Maziyar Panahi, and David Chavalarias,
Gainesville, Fla.: University of Florida Press, 2011, pp. 32–48. “Reconstruction of the Socio-Semantic Dynamics of Political
Activist Twitter Networks—Method and Application to the
de Freitas Melo, Philipe, Carolina Coimbra Vieira, Kiran 2017 French Presidential Election,” PLoS One, Vol. 13, No. 9,
Garimella, Pedro O. S. Vaz de Melo, and Fabrício Benevenuto, September 2018.
“Can WhatsApp Counter Misinformation by Limiting Message
Forwarding?” in Hocine Cherifi, Sabrina Gaito, José Fernendo Giasemidis, Georgios, Nikolaos Kaplis, Ioannis Agrafiotis, and
Mendes, Esteban Moro, and Luis Mateus Rocha, eds., Complex Jason R. C. Nurse, “A Semi-Supervised Approach to Message
Networks and Their Applications VIII, Vol. 1, Proceedings of the Stance Classification,” IEEE Transactions on Knowledge and Data
Eight International Conference on Complex Networks and Their Engineering, Vol. 32, No. 1, January 1, 2020, pp. 1–11.
Applications, London: Springer Nature, 2019, pp. 372–384. Glaser, April, “Russian Trolls Were Obsessed with Black Lives
Derysh, Igor, “Russia Trying to ‘Incite Violence by White Matter,” Slate, May 11, 2018. As of July 29, 2020:
Supremacist Groups’ in US Ahead of 2020 Election: Report,” https://slate.com/technology/2018/05/russian-trolls-are-
Salon, March 10, 2020. As of July 29, 2020: obsessed-with-black-lives-matter.html
https://www.salon.com/2020/03/10/russia-trying-to-incite- Glyer, Christopher, and Nick Carr, “State of the Hack: Spotlight
violence-by-white-supremacist-groups-in-us-ahead-of-2020- Iran—From Cain & Abel to Full SANDSPY,” FireEye, blog post,
election-report/ February 12, 2020. As of September 1, 2020:
Diamond, Larry, and Orville Schell, eds., China’s Influence https://www.fireeye.com/blog/products-and-services/2020/02/
& American Interests: Promoting Constructive Vigilance, state-of-the-hack-spotlight-iran.html
Washington, D.C.: Hoover Institution, October 2018. As of Godson, Roy, and Richard Shultz, “Soviet Active Measures:
September 5, 2020: Distinctions and Definitions,” Defense Analysis, Vol. 1, No. 2,
https://www.hoover.org/research/chinas-influence-american- 1985, pp. 101–110.
interests-promoting-constructive-vigilance
Government of Russia, “Foreign Policy Concept of the Russian
DiResta, Renee, Kris Shaffer, Becky Ruppel, David Sullivan, Federation,” November 30, 2016. As of February 10, 2020:
Robert Matney, Ryan Fox, Jonathan Albright, and Ben Johnson, https://www.mid.ru/en/foreign_policy/official_documents/-/
The Tactics and Tropes of the Internet Research Agency, New asset_publisher/CptICkB6BZ29/content/id/2542248
Knowledge, 2019.
Graphika, “Briefing: China, Iran, and Russia State Accounts on
Doronin, Alexander, “Anti-Extremism: Who Will Be the Target?” U.S. Protests,” blog post, June 3, 2020. As of September 1, 2020:
in Business-Intelligence [Biznes Razvedka], 2010. As of September https://graphika.com/posts/briefing-china-iran-and-russia-state-
10, 2020: accounts-on-u-s-protests/
https://web.archive.org/web/20090621054933/
http://www.agentura.ru/

23
Haigh, Maria, Thomas Haigh, and Nadine I. Kozak, “Stopping King, Gary, Jennifer Pan, and Margaret E. Roberts, “How the
Fake News: The Work Practices of Peer-to-Peer Counter Chinese Government Fabricates Social Media Posts for Strategic
Propaganda,” Journalism Studies, Vol. 19, No. 14, 2018, pp. 2062– Distraction, Not Engaged Argument,” American Political Science
2087. Review, Vol. 111, No. 3, August 2017, pp. 484–501.
Hamilton, Alexander, “The Federalist Papers, No. 68: The Mode Klein, Daniel, “Implementing a General Framework for
of Electing the President,” New York Packet, March 14, 1788 (via Assessing Interrater Agreement in Stata,” Stata Journal, Vol. 18,
Lillian Goldman Law Library). As of July 29, 2020: No. 4, 2018, pp. 871–901.
https://avalon.law.yale.edu/18th_century/fed68.asp
Koc-Michalska, Karolina, Bruce Bimber, Daniel Gomez,
Hanlon, Bradley, “Iran’s Newest Info Op Shows an Evolution of Matthew Jenkins, and Shelley Boulianne, “Public Beliefs About
Tactics,” Disinfo Portal, November 13, 2018. As of September 1, Falsehoods in News,” International Journal of Press/Politics,
2020: Vol. 25, No. 3, 2020, pp. 447–468.
https://disinfoportal.org/irans-newest-info-op-shows-an-
evolution-of-tactics/ Lamond, James, The Origins of Russia’s Broad Political Assault
on the United States, Washington, D.C.: Center for American
Heredia, Brian, Joseph D. Prusa, and Taghi M. Khoshgoftaar, Progress, October 3, 2018.
“The Impact of Malicious Accounts on Political Tweet
Sentiment,” 2018 IEEE 4th International Conference on Lefebvre, Vladimir A., “The Basic Ideas of Reflexive Games
Collaboration and Internet Computing (CIC), conference Logic,” Systems and Structures Research Problems, conference
proceedings, 2018, pp. 197–202. proceedings, Moscow, 1965.

Howard, Philip N., Bharath Ganesh, Dimitra Liotsiou, Lefebvre, Vladimir A., “The Elements of Reflexive Game Logic,”
John Kelly, and Camille François, “The IRA, Social Media Problems in Engineering Psychology, No. 4, 1966.
and Political Polarization in the United States, 2012–2018,” Lefebvre, Vladimir A., “An Algebraic Model of Ethical
Computation Propaganda Research Project, University of Cognition,” Journal of Mathematical Psychology, Vol. 22, No. 2,
Oxford, 2018. 1980.
Ibrishimova, Marina Danchovsky, and Kin Fun Li, “A Machine Lefebvre, Vladimir A., “Reflexive Analysis of Groups,” in Shlomo
Learning Approach to Fake News Detection Using Knowledge Argamon and Newton Howard, eds., Computational Methods for
Verification and Natural Language Processing,” in Leonard Counterterrorism, Berlin: Spring-Verlag, 2009, pp. 173–210.
Barolli, Hiroaki Nishino, and Hiroyoshi Miwa, eds., Advances
in Intelligent Networking and Collaborative Systems: the Lefebvre, Vladimir A., Lectures on Reflexive Game Theory, Los
11th International Conference on Intelligent Networking and Angeles: Leaf & Oaks Publishers, 2010.
Collaborative Systems, London: Springer Nature, 2019, pp. 223–
Lefebvre, Vladimir A., and Jonathan David Farley, “The
234.
Torturer’s Dilemma: A Theoretical Analysis of the Societal
Jang, S. Mo, Brooke W. Mckeever, Robert Mckeever, and Joon Consequences of Torturing Terrorist Suspects,” Studies in
Kyoung Kim, “From Social Media to Mainstream News: The Conflict & Terrorism, Vol. 30, No. 7, 2007, pp. 635–646.
Information Flow of the Vaccine-Autism Controversy in the US,
Lewis-Beck, Michael S., Alan E. Bryman, and Tim Futing Liao,
Canada, and the UK,” Health Communication, Vol. 34, No. 1,
The SAGE Encyclopedia of Social Science Research Methods,
2019, pp. 110–117.
Thousand Oaks, Calif.: Sage Publications, 2004.
Jenkins, Brian Michael, America’s Great Challenge: Russia’s
Lockhart, P. R., “How Russia Exploited Racial Tensions in
Weapons of Mass Deception, WeaponsofMassDeception.net,
America During the 2016 Elections,” Vox, December 17, 2018. As
2019. As of July 29, 2020:
of July 29, 2020:
https://weaponsofmassdeception.net/
https://www.vox.com/identities/2018/12/17/18145075/russia-
Jones, Seth G., “Russian Meddling in the United States: The facebook-twitter-internet-research-agency-race
Historical Context of the Mueller Report,” Center for Strategic
Martin, Diego A., and Jacob N. Shapiro, “Trends in Online
and International Studies, March 2019. As of August 27, 2020:
Foreign Influence Efforts,” Princeton University, version 1.2,
https://csis-website-prod.s3.amazonaws.com/s3fs-public/
July 8, 2019. As of September 1, 2020:
publication/190328_RussianMeddlingintheUS_WEB_V.2.pdf
https://scholar.princeton.edu/sites/default/files/jns/files/trends_
Karjukin, V., and V. F. S. Chausov, “Reflexive Games Counter,” in in_foreign_influence_efforts_2019jul08_0.pdf
V. E. Lepsky, Reflexive Processes and Management [Refleksivnye
Marwick, Alice E., “Why Do People Share Fake News? A
Processy I Upravlyenye], Moscow: Russian Academy of Sciences,
Sociotechnical Model of Media Effects,” Georgetown Law
2017.
Technology Review, Vol. 2, No. 2, 2018, pp. 474–512.
Kavanagh, Jennifer, and Michael D. Rich, Truth Decay: An Initial
Matthews, Miriam, Alyssa Demus, Elina Treyger, Marek N.
Exploration of the Diminishing Role of Facts and Analysis in
Posard, Hilary Reininger, and Christopher Paul, Understanding
American Public Life, Santa Monica, Calif.: RAND Corporation,
and Defending Against Russia’s Malign and Subversive
RR-2314-RC, 2018. As of July 20, 2020:
Information Efforts in Europe, Santa Monica, Calif.: RAND
https://www.rand.org/pubs/research_reports/RR2314.html
Corporation, RR-3160-EUCOM, forthcoming.
Kendall-Taylor, Andrea, and David Shullman, “Converging
Mattis, Peter, “Contrasting China’s and Russia’s Influence
Chinese and Russian Disinformation Compounds Threat to
Operations,” War on the Rocks, January 16, 2018. As of
Democracy,” Power 3.0, May 26, 2020. As of September 1, 2020:
September 1, 2020:
https://www.power3point0.org/2020/05/26/converging-chinese-
https://warontherocks.com/2018/01/contrasting-chinas-russias-
and-russian-disinformation-compounds-threat-to-democracy/
influence-operations/

24
Mazarr, Michael J., Abigail Casey, Alyssa Demus, Scott W. Root, Danielle, and Aadam Barclay, Voter Suppression During
Harold, Luke J. Matthews, Nathan Beauchamp-Mustafaga, and the 2018 Midterm Elections: A Comprehensive Survey of Voter
James Sladden, Hostile Social Manipulation: Present Realities and Suppression and Other Election Day Problems, Washington, D.C.:
Emerging Trends, Santa Monica, Calif.: RAND Corporation, Center for American Progress, November 20, 2018.
RR-2713-OSD, 2019. As of July 22, 2020:
https://www.rand.org/pubs/research_reports/RR2713.html Roth, Yoel, “Information Operations on Twitter: Principles,
Process, and Disclosure,” Twitter Blog, blog post, June 13, 2019.
Michael, George, “Useful Idiots or Fellow Travelers? The As of September 1, 2020:
Relationship Between the American Far Right and Russia,” https://blog.twitter.com/en_us/topics/company/2019/
Terrorism and Political Violence, Vol. 31, No. 1, 2019, pp. 64–83. information-ops-on-twitter.html
Nimmo, Ben, “#TrollTracker: An Iranian Messaging Sanger, David E., and Nicole Perlroth, “Chinese Hackers Target
Laundromat,” DFRLab, April 29, 2018. As of September 1, 2020: Email Account of Biden Campaign Staff, Google Says,” New York
https://medium.com/dfrlab/ Times, June 4, 2020.
trolltracker-an-iranian-messaging-laundromat-218c46509193
Schoen, Fletcher, and Christopher J. Lamb, “Deception,
Niquette, Mark, and Jennifer Jacobs, “China Looks to Disinformation, and Strategic Communications: How One
Influence Iowa in Trade War over Trump Tariffs,” Bloomberg, Interagency Group Made a Major Difference,” INSS Strategic
September 23, 2018. As of September 1, 2020: Perspectives, Vol. 11, June 1, 2012.
https://www.bloomberg.com/news/articles/2018-09-23/china-
looks-to-influence-iowa-in-trade-war-over-trump-tariffs Sebenius, Alyza, “U.S. Sees Russia, China, Iran Trying to
Influence 2020 Elections,” Bloomberg, June 24, 2019. As of
Paul, Christopher, and Miriam Matthews, The Russian “Firehose September 1, 2020:
of Falsehood” Propaganda Model: Why It Might Work and https://www.bloomberg.com/news/articles/2019-06-24/
Options to Counter It, Santa Monica, Calif.: RAND Corporation, u-s-sees-russia-china-iran-trying-to-influence-2020-elections
PE-198-OSD, 2016. As of July 14, 2020:
https://www.rand.org/pubs/perspectives/PE198.html Select Committee on Intelligence of the United States Senate,
Report on Russian Active Measures Campaigns and Interference
Paul, Christopher, and Miriam Matthews, The Language of in the 2016 U.S. Election, Vol. 1: Russian Efforts Against Election
Inform, Influence, and Persuade: Assessment Lexicon and Usage Infrastructure with Additional Views, 116th Congress, Report
Guide for U.S. European Command Efforts, Santa Monica, Calif.: 116-XX, 2019. As of July 29, 2020:
RAND Corporation, RR-2655-EUCOM, 2018. As of July 20, https://www.intelligence.senate.gov/sites/default/files/
2020: documents/Report_Volume1.pdf
https://www.rand.org/pubs/research_reports/RR2655.html
Select Committee on Intelligence of the United States Senate,
Pomerantsev, Peter, “Russia and the Menace of Unreality,” The (U) Report on Russian Active Measures Campaigns and
Atlantic, September 9, 2014. As of August 31, 2020: Interference in the 2016 U.S. Election, Vol. 2: Russia’s Use of Social
https://www.theatlantic.com/international/archive/2014/09/ Media with Additional Views, 116th Congress, Report 116-XX,
russia-putin-revolutionizing-information-warfare/379880/ undated. As of July 29, 2020:
https://www.intelligence.senate.gov/sites/default/files/
Pomerantsev, Peter, “Authoritarianism Goes Global (II): The documents/Report_Volume2.pdf
Kremlin's Information War,” Journal of Democracy, John
Hopkins University Press, Vol. 26, No. 4, October 2015a. Select Committee on Intelligence of the United States Senate,
(U) Russian Active Measures Campaigns and Interference
Pomerantsev, Peter, “The Kremlin’s Information War,” Journal of in the 2016 Election, Vol. 5: Counterintelligence Threats and
Democracy, Vol. 26, No. 4, October 2015b. Vulnerabilities, 116th Congress, Report 116-XX, 2020. As of
Radin, Andrew, Alyssa Demus, and Krystyna Marcinek, August 27, 2020:
Understanding Russian Subversion: Patterns, Threats, and https://www.intelligence.senate.gov/sites/default/files/
Responses, Santa Monica, Calif.: RAND Corporation, PE-331-A, documents/report_volume5.pdf
2020. As of July 14, 2020: Smith, Aaron, Laura Silver, Courtney Johnson, Kyle Taylor and
https://www.rand.org/pubs/perspectives/PE331.html Jingjing Jiang, “Publics in Emerging Economies Worry Social
Radin, Andrew, and Clint Reach, Russian Views of the Media Sow Division, Even as They Offer New Chances for
International Order, Santa Monica, Calif.: RAND Corporation, Political Engagement,” Pew Research Center, May 2019. As of
RR-1826-OSD, 2017. As of August 31, 2020: July 29, 2020:
https://www.rand.org/pubs/research_reports/RR1826.html https://www.pewinternet.org/wp-content/uploads/
sites/9/2019/05/Pew-Research-Center_Technology-and-Politics-
Revelli, Alice, and Lee Foster, “Network of Social Media in-Emerging-Economies-Report_2019-05-13.pdf
Accounts Impersonates U.S. Political Candidates, Leverages
U.S. and Israeli Media in Support of Iranian Interests,” FireEye, Snell, Nicholas, Jeremy Straub, Brandon Stoick, Terry Traylor,
May 28, 2019. As of September 1, 2020: and William Fleck, “Assessing Online Media Reliability: Trust,
https://www.fireeye.com/blog/threat-research/2019/05/ Metrics and Assessment,” in Misty Blowers, Russell D. Hall,
social-media-network-impersonates-us-political-candidates- and Venkateswara R. Dasari, eds., Disruptive Technologies in
supports-iranian-interests.html Information Sciences II, Vol. 11013, International Society for
Optics and Photonics, 2019.
Rogan, Tom, “Russia’s War on Fracking,” National Review,
February 3, 2015. As of July 24, 2020: Soufan Center, “IntelBrief: Russian Disinformation Campaigns
https://www.nationalreview.com/2015/02/ Continue to Sow Doubt in the West,” December 17, 2018. As of
russias-war-fracking-tom-rogan/ July 21, 2020:
https://thesoufancenter.org/intelbrief-russian-disinformation-
campaigns-continue-to-sow-doubt-in-the-west/

25
Spaulding, Suzanne, Devi Nair, and Arthur Nelson, “Russia’s U.S. Department of Justice, “Russian National Charged in
Attacks on Democratic Justice Systems,” Center for Strategic and Conspiracy to Act as an Agent of the Russian Federation Within
International Studies, May 2019. As of July 2020: the United States,” July 16, 2018. As of July 25, 2020:
https://www.csis.org/features/ https://www.justice.gov/opa/pr/russian-national-charged-
russias-attacks-democratic-justice-systems conspiracy-act-agent-russian-federation-within-united-states
Starbird, Catharine, Ahmer Arif, and Tom Wilson, Technical Usdin, Steve, “When a Foreign Government Interfered in a
Proposal: Understanding the Structure and Dynamics of U.S. Election—to Reelect FDR,” Politico, January 16, 2017. As of
Disinformation in the Online Information Ecosystem, Arlington, July 29, 2020:
Va.: prepared for the Office of Naval Research, grant https://www.politico.com/magazine/story/2017/01/
number N00014-17-1-2980, August 14, 2018. As of July 29, 2020: when-a-foreign-government-interfered-in-a-us-electionto-
https://apps.dtic.mil/sti/pdfs/AD1058867.pdf reelect-fdr-214634
Stent, Angela, Putin’s World: Russia Against the West and with Volz, Dustin, “Russia, Iran, North Korea Launch Hundreds of
the Rest, New York: Hachette Book Group, 2019. Cyberattacks on U.S. Political Groups, Microsoft Says,” Wall
Street Journal, July 17, 2019.
Stubbs, Jack, and Katie Paul, “Facebook Says It Dismantles
Disinformation Network Tied to Iran’s State Media,” Reuters, Wallis, Jake, Tom Uren, Elise Thomas, Albert Zhang, Samantha
May 5, 2020. As of September 1, 2020: Hoffman, Lin Li, Alex Pascoe, and Danielle Cave, Retweeting
https://www.reuters.com/article/us-iran-facebook/ Through the Great Firewall: A Persistent and Undeterred Threat
facebook-says-it-dismantles-disinformation-network-tied-to- Actor, Australian Strategic Policy Institute, Policy Brief Report
irans-state-media-idUSKBN22H2DK No. 33, 2020.
Sullivan, Eileen, “Maria Butina, Russian Who Infiltrated Walsh, Gianfranco, and Vincent-Wayne Mitchell, “The Effect of
Conservative Circles, Is Deported,” New York Times, October 25, Consumer Confusion Proneness on Word of Mouth, Trust, and
2019. Customer Satisfaction,” European Journal of Marketing, Vol. 44,
No. 6, 2010.
Taylor, Margaret L., “Combatting Disinformation and Foreign
Interference in Democracies: Lessons from Europe,” Brookings Wang Lin [⦻᷇] and Wang Guibin [⦻䍥┘], “An Analysis of
Institution, blog post, July 31, 2019. As of July 29, 2020: Public Opinion Warfare and Psychological Warfare
https://www.brookings.edu/blog/techtank/2019/07/31/ [㠶䇪ᡈоᗳ⨶ᡈ䗘᷀],” PLA Daily [䀓᭮ߋᣕ], June 8, 2004.
combating-disinformation-and-foreign-interference-in-
democracies-lessons-from-europe/ Ward, Alex, “4 Main Takeaways from New Reports on Russia’s
2016 Election Interference,” Vox, December 17, 2018. As of
Thomas, Elise, and Albert Zhang, COVID-19 Attracts Patriotic July 29, 2020:
Troll Campaigns in Support of China’s Geopolitical Interests, https://www.vox.com/world/2018/12/17/18144523/russia-senate-
Australian Strategic Policy Institute, June 25, 2020. report-african-american-ira-clinton-instagram
Thomas, Timothy L., “Russia’s Reflexive Control Theory and Washington, George, Farewell Address, September 19, 1796, (via
the Military,” Journal of Slavic Military Studies, Vol. 17, 2004, Avalon Project at Yale Law School). As of July 29, 2020:
pp. 237–256. https://www.ourdocuments.gov/
doc.php?flash=true&doc=15&page=transcript
Thomas, Timothy, Russia’s 21st Century Information War:
Working to Undermine and Destabilize Populations, Riga, Latvia: Watts, Clint, “Triad Against Disinformation: How Russia, Iran,
Defense Strategic Communications: The Official Journal of the & China Ally in A Messaging War Against America,” GMF
NATO Strategic Communications Center of Excellence, 2015. As Alliance for Securing Democracy, commentary, May 15, 2020. As
of July 29, 2020: of September 1, 2020:
https://www.stratcomcoe.org/timothy-thomas-russias-21st- https://securingdemocracy.gmfus.org/triad-of-disinformation-
century-information-war-working-undermine-and-destabilize- how-russia-iran-china-ally-in-a-messaging-war-against-america/
populations
Yates, Will, “‘Russian Trolls’ Promoted California
Tingley, Dustin, and Gernot Wagner, “Solar Geoengineering Independence,” BBC, November 4, 2017. As of July 29, 2020:
and the Chemtrails Conspiracy on Social Media,” Palgrave https://www.bbc.com/news/blogs-trending-41853131
Communications, Vol. 3, No. 1, December 2017, pp. 1–7.
Yu Mingsong, “Research on United Front Work for Hong Kong
Tromblay, Darren E., Political Influence Operations: How Foreign Middle Class Professionals [俉⑟ѝӗуъӪ༛㔏ᡈᐕ֌⹄ウ],”
Actors Seek to Shape U.S. Policy Making, Lanham, Md.: Rowman United Front Science, March 2017.
& Littlefield, 2018.
Umpleby, Stuart, “Vladimir Lefebvre’s Theory of Two Systems
of Ethical Cognition,” Journal on Systemics, Cybernetics and
Informatics, Vol. 14, No. 5, 2016, pp. 65–67.
United States v. Internet Research Agency LLC, 2018 W.L. 914777,
2018. As of July 29, 2020:
https://www.justice.gov/file/1035477/download
U.S. Bureau of Citizenship and Immigration Services, Resource
Information Center, Venezuela: Information on Círculos
Bolivarianos (Bolivarian Circles), VEN02001.ZMI, April 30, 2002.
As of September 1, 2020:
https://www.refworld.org/docid/3dec9b4b4.html

26
27
 C O R P O R AT I O N

The RAND Corporation is a research

organization that develops solutions
to public policy challenges to help
make communities throughout
the world safer and more secure,
healthier and more prosperous.
RAND is nonprofit, nonpartisan, and
committed to the public interest.

RAND’s publications do not

necessarily reflect the opinions of
its research clients and sponsors.
About This Report is a registered trademark.
Throughout the U.S. political campaign season of 2020, Russia might try
Limited Print and Electronic
again to manipulate and divide U.S. voters through social media. This report
Distribution Rights
is the first in a four-part series aimed at helping policymakers and the public
understand—and mitigate—the threat of online foreign interference in national, This document and trademark(s)
contained herein are protected
state, and local elections.
by law. This representation of
Given the past and likely extant threats to U.S. elections, the California RAND intellectual property is
Governor’s Office of Emergency Services (Cal OES) asked RAND’s National provided for noncommercial use
Security Research Division (NSRD) for research to help them analyze, fore- only. Unauthorized posting of this
cast, and mitigate threats by foreign actors targeting local, state, and national publication online is prohibited.
elections. Permission is given to duplicate this
document for personal use only, as
We would like to thank our sponsors at the California Governor’s Office of long as it is unaltered and complete.
Emergency Services (Cal OES). We are grateful to Melissa Bauman, whose dedi- Permission is required from RAND
cated work improved the prose of this report. Finally, we thank Christopher Paul to reproduce, or reuse in another
of RAND and Herbert Lin of Stanford University for their thoughtful reviews. form, any of our research documents
for commercial use. For information
This research was sponsored by the California Governor’s Office of Emergency on reprint and linking permissions,
Services (Cal OES) and conducted within the International Security and Defense please visit www.rand.org/pubs/
Policy Center of the RAND National Security Research Division (NSRD). NSRD permissions.
conducts research and analysis for the Office of the Secretary of Defense, the
Joint Staff, the Unified Combatant Commands, the defense agencies, the Navy, For more information on this
publication, visit
the Marine Corps, the U.S. Coast Guard, the U.S. Intelligence Community,
www.rand.org/t/RRA704-1.
allied foreign governments, and foundations.
© 2020 RAND Corporation
For more information on the RAND International Security and Defense Policy
Center, see www.rand.org/nsrd/isdp or contact the director (contact information
is provided on the webpage). www.rand.org

RR-A704-1