TECHNICAL EXPLOITATION IN THE GRAY ZONE: EMPOWERING NATO SOF FOR STRATEGIC EFFECT by Chace A. Falgout A Capstone Project Submitted to the Faculty of Utica College May 2019 in Partial Fulfillment of the Requirements for the Degree of Master of Science in Cybersecurity      ProQuest Number: 13864931       All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion.       ProQuest 13864931 Published by ProQuest LLC (2019 ). Copyright of the Dissertation is held by the Author.   All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code Microform Edition © ProQuest LLC.  ProQuest LLC. 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, MI 48106 - 1346 © Copyright 2019 by Chace A. Falgout All Rights Reserved ii Abstract Russian hybrid warfare has become the principle threat to NATO over the last decade. From the Baltic Sea to the Black Sea, Russia has exercised its will across Europe; inciting tensions while limiting its activities to below the Article 5 threshold, an armed attack on one is an attack on all. The balance of power favors those who embrace inevitable technological advancement while enduring the discomfort presented by its evolution. Hybrid warfare creates complex problems requiring an unconventional mindset and while NATO Special Operations (NSOF) inherently possess this trait and are rightly-suited to contribute to NATO’s counter-hybrid strategy, little research examines how NSOF tactical activities can deliver strategic effects through the exploitation of technology. This capstone collates expansive research on Russian gray zone activities of hybrid warfare, NATO’s cyber deterrence and counter hybrid threat strategy, and NSOF’s doctrine and capabilities, to present a focused area for capability enhancement of Special Operations Forces (SOF). NSOF’s embrace of Technical Exploitation Operations (TEO) facilitated evidence-based operations in Afghanistan with great success and subsequently led to the establishment of TEO programs in Alliance nations. However, sub-disciplines like biometrics have received preference over data-rich sources like digital media, cellular phones, and other more complex exploitation forms. An appreciation of the value of digital artifacts and their ability to illuminate hybrid warfare and gray zone activities, intent, and attribution is necessary to accurately position NSOF in NATO’s cybersecurity and hybrid warfare framework. Keywords: Cybersecurity, Dr. Christopher Riddell, hybrid warfare, gray zone activities, drone forensics. iii Acknowledgements This Capstone would not have been possible without the support of my wife and family. I would like to thank Dr. Jeffrey V. Gardner, Dr. Christopher Riddell, and Dr. Stephen F. Pearson for their guidance and technical support during this process and Mr. Mark Low, Kally Lange, and Sam Pepenella for their review and suggestions. iv Table of Contents List of Illustrative Materials........................................................................................................... vi DISCLAIMER .............................................................................................................................. vii Technical Exploitation in the Gray Zone: Empowering NATO SOF for Strategic Effect ............. 1 Literature Review.......................................................................................................................... 12 Russian use of Gray Zone Activities in Hybrid Warfare .......................................................... 12 State-sponsored advanced persistent threats (APT). ............................................................. 18 Critical infrastructure. ........................................................................................................... 20 Cyber crime and organized crime. ........................................................................................ 23 Information operations (IO). ................................................................................................. 25 Subverting democracy. ..................................................................................................... 28 Electronic warfare ............................................................................................................. 30 Compromised software and hardware. ................................................................................. 33 NATO’s Approach to Current & Future Challenges ................................................................ 33 Hybrid warfare strategy ........................................................................................................ 35 Cyber strategy. ...................................................................................................................... 38 Tactical to strategic cyber intelligence requirements........................................................ 41 Technical exploitation strategy ............................................................................................. 46 NATO SOF & NSHQ ............................................................................................................... 50 The Special Operations Component Command (SOCC)...................................................... 52 Optimizing Technical Exploitation for the Gray Zone ............................................................. 54 Mobile device & computer exploitation. .............................................................................. 56 Network exploitation. ........................................................................................................... 58 The internet of things (IoT)................................................................................................... 60 Drone exploitation. ............................................................................................................... 61 Vehicle entertainment & navigation exploitation. ................................................................ 64 Multi-level information sharing. ........................................................................................... 66 Technical exploitation and attribution .............................................................................. 69 Discussion of the Findings ............................................................................................................ 70 Anticipating Gray Zone Activities ............................................................................................ 71 Revising Doctrine ..................................................................................................................... 76 NSOF leads CHSTs. ............................................................................................................. 79 Integrating Exploitation Capabilities ........................................................................................ 80 Strategic and operational support.......................................................................................... 80 TEO within the CHSTs. ........................................................................................................ 82 Exploitation analysis ............................................................................................................. 83 Limitations of the Study ........................................................................................................... 84 Future Research and Recommendations ....................................................................................... 85 New Research Question 1: ........................................................................................................ 86 New Research Question 2: ........................................................................................................ 86 New Research Question 3: ........................................................................................................ 87 Recommendations ......................................................................................................................... 88 References ..................................................................................................................................... 90 Appendix A: Forensic Questions to Reduce Assumptive Associations ..................................... 137 v List of Illustrative Materials Figure 1. The spectrum of conflict in unconventional warfare ....................................................... 2 Table 1. Hybrid Tools .................................................................................................................. 13 Figure 2. The cyber kill chain ....................................................................................................... 20 Figure 3. Differing national levels of threat perception ................................................................ 35 Figure 4. Cyber intelligence: Responsibilities and interrelations ................................................. 42 Figure 5. Gray zone and hybrid tools with distribution ................................................................ 88 vi DISCLAIMER THE OPINIONS AND CONCLUSIONS EXPRESSED HEREIN ARE THOSE OF THE AUTHOR AND DO NOT REPRESENT THE VIEWS OF ANY POLITICAL, MILITARY, ACADEMIC, OR COMMERCIAL ORGANIZATION. THE RESEARCH HEREIN WAS CONDUCTED ON THE INTERNET WITH PRECAUTION NOT TO DETAIL INFORMATION WITH CLASSIFICATIONS: NATO UNCLASSIFIED AND UNITED STATES FOR OFFICIAL USE ONLY. WHILE MOST OF THESE DOCUMENTS ARE OPENLY AVAILABLE ON THE INTERNET AND THEIR THOROUGH EXAMINATION WOULD HAVE AMPLIFIED THE RESULTS AND SUPPORTED FUTURE RESEARCH RECOMMENDATIONS, THE AUTHOR CHOSE TO RESPECT DOCUMENT DISSEMINATION RESTRICTIONS AND PROTECT ALLIED INFORMATION. vii Technical Exploitation in the Gray Zone: Empowering NATO SOF for Strategic Effect Article 5 of the North Atlantic Treaty is abbreviated as; an armed attack on one is an attack on all and is the heart of the North Atlantic Treaty Organization’s (NATO) collective defense strategy (1949). Although Article 5 was invoked only once, following the September 11, 2001 attacks on the United States, the threshold for its invocation was questioned following Russian aggression in Estonia (2007), Georgia (2008), and Ukraine (2014). Upon acceptance into NATO in 2004, Estonia emphasized the importance of cyber defense and proposed a cyber defense center in the Estonian capital, Tallinn. The Supreme Allied Commander Transformation approved the concept in 2006, and in May 2008, The Collaborative Cyber Defense Centre of Excellence (CCDCOE) was established as an International Military Organization (CCDCOE, 2019a). Estonia’s request and foresight were justified, in April 2007, Estonia endured three weeks of cyber attacks, specifically Distributed Denial-of-Service (DDoS) attacks on Internet banking infrastructure, which accounted for over 97% of Estonian banking (Geers, 2009). The Estonian government did not regard these events as an armed attack, but instead classified them as cyber crimes below the Article 5 threshold (Tikk, Kaska, & Vihul, 2010). According to Hoffman (2016), nations without the conventional means to realize strategic goals obstruct international response by employing “ambiguously aggressive actions” (p. 26). The gray zone is the area where state and non-state actors utilize tactics below the threshold of war to promote military and political agendas (Roberts, 2016). Hoffman (2018) later defined gray zone tactics as: those covert or illegal activities of non-traditional statecraft that are below the threshold of armed organized violence; including disruption of order, political sub-version of government or non-governmental organizations, psychological operations, abuse of legal 1 processes, and financial corruption as part of an integrated design to achieve strategic advantage. (p. 36) Counter-gray zone activities are relevant SOF missions because they occur under the Article 5 threshold and do not trigger the activation of resources to overwhelm an adversary with conventional force (Moon, 2018). Gray zone activities are indirect and aim to circumvent violence while hybrid warfare includes multi-dimensional aggression consisting of conventional and unconventional tactics (Hoffman, 2018; Kofman & Rojansky, 2015; Mazar, 2015). The purpose of this research was to examine the role of technical exploitation in nonArticle 5 crisis response operations, specifically NSOF’s use of tactical digital forensics to provide operational and strategic intelligence to counter gray zone activities. What are NATO’s requirements for collecting cybersecurity information? How should NSOF optimize exploitation’s procedures and tools to identify forensics artifacts relevant across the operational spectrum? What are the benefits of a framework for cross-operational sharing of cybersecurity and digital forensics information? Hoffman’s 2007 introduction to hybrid warfare consolidated ideas within the United States’ defense community and explained the benefit of combining multiple modes of war with deliberate design for a particular operational environment. Hybrid warfare replaces tanks with propaganda, political and social agitation, cyber attacks and other non-lethal tools to impose national will (Schnaufer, 2017). Hoffman placed gray zone activities at the far left of a spectrum of conflict, more ambiguous than irregular and hybrid warfare (see Figure 1). Figure 1. The spectrum of conflict in unconventional warfare (adapted from Hoffman, 2016, p. 29). 2 On August 7, 2008, escalating tensions between Georgia and Russian-supported separatists prompted the Georgian Army to attack dissidents in the South Ossetia (Tikk et al., 2008). The same day, Russia launched DDoS attacks and defaced Georgian government websites, news and media sites, and Georgia’s largest bank, in a similar fashion to the 2007 attacks on Estonia (Tikk et al., 2008). On August 8, Russian ground forces entered Georgian territory, swiftly overrunning Georgian positions in South Ossetia and Abkhazia (Pruitt, 2018). Although Russia celebrated a tactical victory, not all aspects were celebrated equally. Following the 2008 five-day conflict between Russia and Georgia, information warfare theoretician Igor Panarin called for reallocation of funds to increase Russia’s international propaganda and information warfare capabilities and described Russia’s inability to defend state “goals and interests in the global information space” (as cited in Lysenko & Brooks, 2018, para 37). Cyberspace negotiations between Russia and the United States took place a year later, in 2009, but produced contrary approaches; the deputy secretary of Russia’s Security Council demanded a ban on covertly embedded code or circuitry in commercial electronics, while the United States favored improving international law enforcement (LE) cooperation (Markoff & Kramer, 2009). Strategic desires of the two nations were not overcome and the talks ended with the United States arrogantly suggesting Russia address its cybersecurity concerns by developing layered defenses (Shuster, 2018). In 2009, the United States military cyber responsibilities were handled as secondary responsibilities of unified commands and joint task forces. The United States National Cybersecurity Strategy was not fully accomplished, with key deficiencies in cyber indications and warning, infrastructure control systems, and cyber crime (Powner, 2009). After a decade of transition, the United States Cyber Command received initial operational capability status as a 3 sub-unified command in 2010, with elevation to a Unified Combatant Command in August 2017 (United States Cyber Command, 2019). NATO also made progress with cybersecurity initiatives in 2009 when the NATO CCDCOE began work on the Tallinn Manual 1.0, an objective description of international law governing cyber warfare, not to be confused with best practices, doctrine, or the promotion of change (Schmitt, 2013). Both Russia and China share like-minded strategies on cyber warfare, information security and their implementation, viewing them as components of holistic information operations (IO) (Giles & Hagestad, 2013). Russia and China have also heavily invested in new technologies in attempt to reduce the economic and power gap between themselves and their Western counterparts (Polyakova & Boyer, 2018). According to the Office of the National Counterintelligence Executive (2011), Russia’s self-perception as a strategic rival of the United States guides their collection effort toward “U.S. economic information and technology” (p. 12). This self-perception is also consistent with The Assistant Secretary of Defense for Research and Engineering Mary Miller’s description of China’s ambition to match United States’ microelectronics capabilities by 2020, with intent to dominate the global market by 2030 (as cited in Cronk, 2018). In the last decade, Russia used these advanced cyber capabilities to infiltrate and manipulate political and economic centers of gravity as forms of psychological terrorism (Connell & Vogler, 2017). In a televised interview in 2013 Russian Defense Minister Sergei Shoigu claimed Russia possessed “weapons of mass destruction in cyberspace” (as cited in Shuster, 2018, para 14). The CEO of recognized Russian cybersecurity company NPO Echelon Alexy Markov Shoygu, announced the commissioning of the Russian Cyber Command on January 14, 2014, while recognizing the Ministry of Defense’s website did not confirm its existence (Markov, 2014; as 4 cited in Lysenko & Brooks, 2018). Russia continued its cyber capability expansion and in 2015 the United States Director of National Intelligence James Clapper stated, “the Russian cyber threat is more severe than we had previously assessed” (Clapper, 2015; as cited in Shuster, 2018, para 15). That same testimony to the United States Senate Armed Services Committee named Russia as the greatest cyber threat to the United States (Clapper, 2015; as cited in Gady, 2015). By 2010, the United States Department of Defense (DoD) had eight expeditionary forensic laboratories supporting NATO operations in Afghanistan, and soldiers conducting technical exploitation operations (TEO) in conjunction with these labs greatly enhanced the quality and quantity of raw information from the operational environment (Vores, 2012; Herion, 2012). NATO defines exploitation as “taking full advantage of any information that has come to hand for tactical or strategic purposes” (NATO Standardization Office [NSO], 2018a, p. 49). NATO doctrinal standards on exploitation were outlined in 2015, in Allied Intelligence Publication 10 (AIntP-10) on Technical Exploitation (NSO, 2015), which designated biometrics, forensics and document and media exploitation (DOMEX) as its pillars. Sub-categories of the DOMEX pillar are document exploitation (DOCEX), media exploitation (MEDEX), and cellular phone exploitation (CELLEX) (Morris, 2016). Human identity is an amalgamation of various biologic, physical, and behavioral traits and the combination of these data with traditional intelligence information is called Identity Intelligence (I2) (Morris, 2016). Conducting TEO facilitates the link between individuals and materials recovered during tactical missions and is a key component of I2 (Lunan, Moore, Moore, & ter Horst, 2018). Although I2 is not a recognized NATO term, NATO exploitation doctrine is built on United States exploitation architecture outlined in various DoD publications emphasizing the utility of exploitation and I2, e.g. Joint Publication 2-0 (2013), and Joint 5 Publication 3-05 (2014) (as cited in Aftergood, 2014). Sharing of I2 occurs between various international organizations including NATO, the Organization of Security Co-operation in Europe (OSCE), European Police Office (Europol), and other United States military commands (Morris, 2016). United States doctrine on exploitation and I2 is found in Army Training Publication 3-90 (2015) and Joint Doctrine Note on Identity Activities 2-16 (2016). Exploitation and the application of I2 in Iraq and Afghanistan catalyzed the targeting cycle, promoted deterrence by minimizing the utility of the adversary’s new technical capabilities, provided evidence of state sponsorship of hostilities, and supported strategic planning (Smith, Thomas, & Tranchemontagne, 2014). Early adopters of military exploitation avoided using the term evidence when related to tactical collection because intelligence derived from tactical exploitation was used in the intelligence cycle, not a court of law (Pearson, 2018). The adversary’s low-use of technology in the Iraq and Afghanistan wars promoted biometrics and contributed to the divide between evidence-based, digital artifact collection and tactical collection (Pearson, 2018). NATO made progress with the biometrics domain of exploitation through NATO standardization agreement 4715, which describes interoperability requirements and the creation of the NATO Automated Biometrics Information System (NABIS) (Niculescu & Coman, 2017; NSO, 2013a). In addition, the NSO (2016) published Allied Intelligence Publication 15, Countering threat anonymity: Biometrics in support of NATO operations and intelligence (AIntP-15) to standardize the collection, storage, and matching of biometric data. Lunan et al., (2018) discussed the positive correlation between effective biometrics support to military operations to the size of its related database and quality of its information architecture. NATO’s answer to a biometric information architecture was the development and employment of the NATO Automated Biometrics 6 Identification System (NABIS) which “will give commanders the ability to more quickly and accurately discover, identify, and record the identities of threat actors” (Lunan et al., 2018, p. 38). Pearson (2013) argued the importance of avoiding assumptive associations between technical artifacts, like user accounts, and people, by promoting a complete “technology to biology bridge” (para 3). Because LE training centers align detailed evidence collection and preservation procedures with the scrutiny of a courtroom, it not uncommon to find more cases of assumptive associations from exploitation and forensic personnel trained outside of LE centers (Pearson, 2013). While intelligence analysis seldom provides a complete picture and some assumptions are expected, Pearson (2013) recognized the validity of applying evidence-based collection procedures to SOF and suggested combining elements from LE collection to timecompressed SOF TEO to clarify the “technology to biology bridge” (para 3). Braccini et al., (2016) provided a comprehensive framework for SOF conducting digital forensics, concentrating on the collection and preservation of digital artifacts in hostile environments. Among the recommendations of Braccini et al., (2016) was the idea that cyber intelligence prior to TEO could reveal critical information about an adversary’s network, infrastructure, or actors. Mancini, Monti, and Panico (2017) expanded on previous work and presented an argument justifying that data collected without exact forensic standards does not automatically negate its evidentiary value. McCulloh and Johnson (2013) suggested three operational imperatives to counter hybrid warfare activities: a) focus operational art on the disruption of the hybrid activity’s logic, not simply attending to the physical actions and their immediate objectives, b) establish clear connections between tactical action to strategic aim, and c) avoid linear, repeated responses 7 within a particular geographic space for an extended period. Russia demonstrated the quick application of lessons learned from the Estonian and Georgian conflicts when they unveiled improvements to their hybrid warfare strategy in Ukraine. These revisions provided justification and legitimacy for Russian presence in Ukraine while simultaneously exploiting vulnerable infrastructure, financial markets, and ethnic grievances without invoking conventional war (Fryc, 2016). Habeck and Harrison (2016) noted that Russian hybrid strategy was guided by trend analysis and the deliberate and precise employment of force to task to avoid predictability. Months prior to the annexation of Crimea, Russia began IO on Russian language television and radio stations warning of the disadvantages of abandoning Russian support in search for a closer European partnership (Kofman et al., 2017). In February 2014, Russia diverted the attention of the international community away from Ukrainian civil unrest by carrying out a large exercise followed by cyber attacks, communications jamming, and the infiltration of Russian Special Forces into the Crimean Peninsula (Moon, 2018). According to a recent poll from the Russian non-governmental research organization, The Levada-Center, 45% of Russians viewed the annexation of Crimea as reason for national pride (as cited in Statista, 2019). Raugh (2016) noted that while these hybrid threats were immediately and directly felt in Ukraine, they would grow to become the greatest threat to the West due to the probability of their continued use and the implication of their consequences. In 2014, Long suggested NSOF would be the principle component of European counterhybrid warfare campaigns by combining small, highly skilled units with advanced exploitation technology. A paradigm shift from conventional warfare to hybrid warfare increased demands on nations trying to sustain conventional forces and work to defend against hybrid activities. In 2015, the United States Chairman of the Joint Chiefs of Staff described the resource challenges 8 of simultaneously protecting the United States and its Allies against near-peer threats and nonstate actors (Dempsey, 2015; as cited in Garamone, 2015). In addition to the aforementioned benefits, NSOF are inexpensive compared to conventional assets like submarines, and capitalizes on combined training facilities and equipment standardization (Long, 2014; Moon, 2018). NATO reaffirmed its 2014 Wales dedication to improving its cyber capabilities during the Warsaw summit of 2016, when it announced cyber defense as a core task of collective defense and an operational domain, whereby cyber activities could be employed to respond to hybrid threats (NATO, 2016). The CCDCOE international relations director Siim Alatalu (2017) suggested a model NATO cyber command would a) replicate the coordination mechanisms of the NATO Special Operations Headquarters (NSHQ), b) possess a civilian computer incident response capability, and c) act as a clearinghouse for sharing tactical and operational cyber defense information (as cited in Pomerleau, 2017). NATO’s priorities following the 2018 Brussels Summit were: a) increased conventional force readiness, b) intelligence reform to include increased information sharing related to hybrid warfare, cyber, and terrorism, c) a Cyberspace Operations Centre in Belgium, to coordinate NATO operations in cyberspace, and d) enhanced energy security through NATO supported, critical infrastructure protection against cyber attacks (Kramer, Binnendijk, & Speranza, 2018). It is important to note, while one of the four stated priorities was enhanced energy security, that topic was listed as item, 78 of 79 on the Brussels Summit Declaration (NATO, 2018e). Although increasing NATO conventional military power was a priority, the RAND Corporation concluded that Russian military capacity has not sufficiently improved as to provoke near-peer conflict without transparent threat to its sovereignty (Boston & Massicot, 2017). 9 The Brussels Summit mirrored the Final Report of the United States Defense Science Board (DSB) Task Force on Cyber Deterrence (2017), which emphasized critical infrastructure and admitted the United States would be unable to defend critical infrastructure against cyber attacks from adversaries within the next decade. This was later corroborated by classified Pentagon scenarios that overwhelmingly acknowledge the likelihood of an initial cyber attack on United States civilian infrastructure in future conflicts (Sager, 2018). The principle finding of the DSB Task Force (2017) was the improvement of attribution capabilities by a) improving user authentication, b) cross-domain/cross-platform information sharing, and c) associating actions to actors through behavioral analysis. The DSB Task Force on Cyber Deterrence (2017) identified three distinct categories of cyber deterrence challenges as: a) the growing cyber capabilities of Russian and China, b) the ability of Iran and North Korea to use or modify commercial cyber tools to attack United States critical infrastructure, and c) the growing capacity for non-state actors to execute low-grade, high-volume cyber attacks. The DoD plan to address these cyber deterrence challenges included tailored deterrence campaigns, increasing cyber resilience, and enhancing existing capabilities (DSB Task Force on Cyber Deterrence, 2017). Retired United States General Stanley McChrystal (2011) popularized, “it takes a network to defeat a network,” a mantra that led Joint Special Operations Task Force (JSOTF) counterinsurgency efforts in the Iraq War. In fact, McChrystal operationalized concepts of netwar presented by Arquilla and Ronfeldt, who used the term to describe future modes of conflict consisting of small disparate organizations attuned to technology, the information age, and which operated with a high level of autonomy (1996). Countering such networks is challenging because their offensive and defensive activities cross physical, political, and societal boundaries while integrating civilian, criminal, military, and LE (Arquilla & Ronfeldt, 1996). 10 Shea (2017) related the importance of the network concept to NATO’s cyber defense strategy by suggesting that successful cyber partnerships require attention, sustained relationships, and collaboration to manage more activities, more actors, and greater strategic risk. European cooperation and networking on cyber crime began in 2001, with the signing of the European Union Convention on Cybercrime Treaty 185, of which Russia and China are not signatories (Council of Europe, 2001). The complexities of prosecuting international cyber crime quickly followed with initial research highlighting the importance of cooperation between international LE (Brenner & Schwerha, 2002). As of 2017, 22 of the 29 NATO member countries signed a memorandum of understanding (MOU) to share cyber-related information to include lessons learned and cyber intelligence (Shea, 2017). Ellis, Black, and Nobles (2017) suggested SOF’s role should be modified to emphasize strategic effects and included decreased focus on lethal operations and increased capabilities in IO, cyber, and civil affairs. DeTrinis (2017) suggested a permanent NSOF presence engaged in a comprehensive military assistance role in vulnerable areas of the Baltics would counter Russian recruitment efforts to establish a baseline for an insurgency. Moon (2018) reiterated that although SOF were the best fit to counter an adversary’s gray zone attacks, neither the NSHQ nor NATO SOF are adequately resourced to attend to the existing security environment, and SOF’s role in cyber and gray zone to hybrid activities should be clearly defined and limited. Research on Russian hybrid warfare strategy and application of gray zone tactics are extensive and would constitute volumes of text. In addition, research on digital forensics is too plentiful to mention in the references of this Capstone, but the release of AIntP-10 on Technical Exploitation (NSO, 2015) and the publication of Battlefield Digital Forensics, by Braccini et al. (2016) laid the framework for solid exploitation collection and preservation. However, little 11 research discusses the particular digital artifacts that could be found through exploitation of gray zone activities, hybrid warfare tools, and their relation to Russian strategic goals. NSOF possess the baseline skills and capabilities to identify digital artifacts indicating Russian intent and protocols but require guidance, support, and infrastructure to facilitate triage and subsequent analysis. This research does not provide a comprehensive solution for countering Russian hybrid threats or exploiting gray zone activities but amplifies the existing body of literature to shape future policy, organization, and techniques to elucidate Russian intent through the exploitation of digital artifacts. Literature Review Russian use of Gray Zone Activities in Hybrid Warfare In 2013, Russian General of the Army Valery Gerasimov published a 7-page monograph outlining the challenges and future forms of war to include non-combat, whole-of-government activities (Gerasimov, 2013). In fact, Gerasimov referred to this description as modern warfare, not hybrid warfare, although scholars and military leaders in the United States had used the latter description for years (Steder, 2016). The strength of a Gerasimov-led hybrid war is the ability to create an unseen enemy whose presence cannot be confirmed, but this delicate symphony is vulnerable to exposure when attention is directed to the connection between tactics and national goals (McKew, 2017). Gray zone activities are often listed as hybrid warfare tools but can occur in the absence of conventional forces and without armed conflict (Hoffman, 2018). This research maintained the original definitions of the references used herein, while recognizing the discussed activities could be employed in the gray zone or as a component of hybrid warfare. The focus is on the digital artifacts exploited from these activities and their strategic value, not the accuracy of their classification. 12 Treverton (2018) listed hybrid threat tools with short examples to simplify their relation to intelligence challenges (see Table 1). Table 1 Hybrid Tools Note: Hybrid tools (Treverton, 2018, p. 10). Hybrid war is executed on at three fronts: the traditional battlespace, the local communities near the conflict, and the international population (Hoffman, 2007). Countering hybrid activities requires knowledge, capabilities, and the flexibility to transition between asymmetric and conventional tactics across the operational spectrum (Deep, 2015). Practitioners of hybrid warfare utilize transnational organized crime networks’ human, logistic, and financial capital to maximize hybrid effects while minimizing their direct resource consumption (Schroefl, & Kaufman, 2014). Russian political and military activity over the last five years indicates ambition to destabilize NATO and modify the European security environment (Bartles, 2017). Bugajski (2016) suggested Russia is motivated to transform parts of Europe into Russian 13 appendages while subverting European connection to North America. Thornton (2017) furthered these claims stating the desired result of Russian military strategy was to create “an incapacitating disunity – both within targeted states within their alliances” (para 11). Following the removal of pro-Russian Ukrainian President Viktor Yanukovych in February 2014, Russia launched a small footprint of SOF to capture key areas of the Crimean Peninsula (Kofman et al., 2017). Russia capitalized on the interim Ukrainian government’s decision to remove Russian as the official language and distributed propaganda describing the act as an existential threat to the Russian-speaking people of eastern Ukraine and Crimea Kofman et al., 2017). The inarticulate delivery of pro-Russian propaganda prior to and following the annexation of Crimea suggested the decision to invade was impromptu; however, the arrival of conventional forces after SOF secured key terrain suggested the idea was properly strategized (Kofman et al., 2017). The invasion of Crimea was also accompanied by synchronized cyber attacks and communications jamming with all collective tactics remaining below the threshold of war (Moon, 2018). The deficiencies in IO noted during the conflict in Georgia in 2008 were remedied and post invasion propaganda was expanded to disseminate alternate history and exploit language, culture, and nationalism (Giles, 2016). Russia also escalated their cyber offensive to critical infrastructure and cut power to over 200 thousand Ukrainian citizens in December 2015 (Lee, Assante, & Conway, 2016). According to Oleh Derevianko, a cybersecurity expert working with the Ukrainian government, Ukrainian critical infrastructure and political and election networks were constantly being probed to both sabotage and test response times and countermeasures (as cited in Cerulus, 2019). Russia’s military strategy is to possess sufficient capability to be perceived as a threat and deter NATO from interfering with Russian affairs, not to win a protracted conflict against NATO 14 (Kyle, 2019; Thornton, 2017). Russian conventional military power is concentrated in areas with the highest probability of engaging the United States and its NATO allies, in the border areas between Allied countries and former Soviet Union states. In contrast, the United States and NATO do not have sustained, large forces in adjacent regions (Bartles, 2017). Over 280,000 United States troops have withdrawn from Europe since the 1980s and it is unlikely that a NATO force at any state of readiness would be able to prevent a Russian incursion along its eastern flank (Erlingsson, 2018). Former NATO Military Committee Chairman Petr Pavel claimed the size of Russia’s army enables them to invade and overpower Estonia, Latvia, and Lithuania within two days regardless of their NATO status (as cited in Bugajski, 2016). In attempt to level the balance of conventional forces in these regions, NATO deployed four, multinational battlegroups to Poland, Lithuania, Latvia, and Estonia, and allowed Russian arms control inspectors to visit some sites as a demonstration of transparency (NATO, 2019a). These battlegroups would not stop a Russian invasion but affirm the Alliance’s Article 5 pledge with “trip-wire deterrence” (Kyle, 2019, p. 112). Boston and Massicot (2017) asserted Russian leadership fully understood the disadvantages of a near-peer conflict with NATO and no indications suggested Russia would deliberately violate NATO’s Article 5 principles. Russian military doctrine, structure, and exercises are centered on defense as opposed to power projection (Boston & Massicot, 2017). While an acute vulnerability is felt in the Baltic States, due to proximity and disproportion to Russian military capabilities, this concern is not shared by other European governments who are confident in NATO’s overwhelming military superiority (Larrabee et al., 2017). Author and Military Officer Sándor Fábián (2016) analogized Russia’s hybrid behavior as a boxer feigning punches, where showing the intent of conventional action in specific areas in Europe instigates 15 NATO leaders to support resource-intensive countermeasures while Russia waits to deliver a hybrid left hook (Fábián, 2016). Putin views cyber warfare as a vital and proven capability to disrupt Western opposition to Russian solidarity (Carstens, 2017). Russian success in Ukraine was largely due to a combination of complex IO and the innovative use of SOF, which suggests future counter-hybrid strategies include pragmatic and technically superior responses across the operational spectrum (Perry, 2016). Considering the reality of Russia’s conventional military capabilities, they are likely to employ the following strategies if engaged in direct conflict: a) utilize asymmetric tactics over several domains to deter escalation and promptly dissolve conflict, b) direct kinetic action and cyber and electronic warfare on adversary command and control (C2) nodes, and c) highvolume, high-accuracy, and highly mobile indirect fire (Boston & Massicot, 2017). Russia’s offensive action will likely focus on adversary headquarters and communications nodes via cyber attacks, the rapid deployment of small, elite forces specializing in attack and maneuver, and the use of deception and concealment within the population (Black Hat Europe, 2017). One of three of the anticipated scenarios of Russian aggression in the Baltics includes limited incursions in Baltic territory, but heavy promotion of dissent of large Russian-speaking population (Larrabee et al., 2017). The ethnic Russian population in Estonia and Latvia is 24% and 25%, respectively with the population of Lithuania reporting 5% ethnic Russians (Kyle, 2019). Although Russia may seek to persuade ethnic Russians and the Russian speaking population in the Baltics, Grigas (2016) noted fundamental differences between ethnic Russians in Ukraine and those in the Baltics and concluded it is unlikely an intense Russian propaganda campaign will motivate the latter demographic toward the Kremlin’s foreign policies (Grigas, 2016). 16 Russian hybrid warfare includes increased human intelligence (HUMINT) assets in the field, especially in non-Russian countries with a large Russian-speaking population (Juurvee, 2018). Latvia, Estonia, and Germany have seen an increase in martial arts schools teaching Russian Systema, a form of martial arts promoted by the Russian military, in addition to shooting and paintball clubs teaching Russian small-arms tactics and openly displaying Russian military insignias (Applebaum, 2018a). Unfortunately, the United States’ surveillance and reconnaissance activities over the last two decades have heavily relied on technical methods in lieu of HUMINT, which contributed to several intelligence failures in the 21st century (Margolis, 2013). Russia has also used private military companies (PMCs) to collect HUMINT and support Syrian government forces in combat against the United States and anti-Syrian regime forces. On February 7, 2018, a United States-led coalition fought and killed up to 300 enemy in Deir alZour, Syria; subsequent intelligence indicated many of the enemy consisted of members of the Wagner Group, a Russian PMC (Linder, 2018). Cleveland, Linder, and Dempsey (2016) noted the ability of adversaries to circumvent Western military strengths within the human domain, and suggested amendments to existing doctrine and the titling of SOF as the principle implementation vehicle to counter gray zone activities. Information collected by the Russian intelligence services (RIS) contributes to the planning and execution of IO (Juurvee, 2018). In March 2014, the RIS used signals intelligence (SIGINT) to intercept a conversation between the European Union Representative for Foreign Affairs and Security Policy and the Estonian Minister of Foreign Affairs, and subsequently post the content on Russian media companies like Russia Today (RT) (Juurvee, 2018). While the Western militaries and governments take precautions to secure communications and operational 17 information, citizens, public leaders, non-profit organizations, and smaller businesses remain vulnerable to the advanced HUMINT and SIGINT capabilities of the RIS (Juurvee, 2018). NATO and the United States practice a minimal level of operational security to safeguard the specifics of military activities, while Russia generously discusses new strategies and equipment in open-source media (Bartles, 2017). According to Grau and Bartles (2016), Russian military decision-making allows more autonomy to lower-level unit commanders, unlike the United States Army’s heavy reliance on staff support. Russian tactical commanders’ autonomy permits variations in personal experience and opinion and suggests the pervasiveness of alternative courses of action in lieu of those anticipated by Western staffs (Grau & Bartles, 2016). The use of hybrid war is prioritized around targeting national or organizational vulnerabilities with non-lethal, socio-political tools (Bartosh, 2018). In March of 2013, Russian President Vladimir Putin announced the use of information attacks to solve political and military problems stating information yields superior results to the use of conventional weapons (Lysenko & Brooks, 2018). Russia has a vested interest in cyber collection of sensitive information to advance political, economic, and military agendas and could do so from behind Russian borders or within Allied territory (Geers, Kindlund, Moran, & Rachwald, 2015). Research from the RAND Corporation suggested Putin’s reaction to political confrontation includes conflict escalation supporting strategic objectives, but in unexpected and largely non-attributable means, as indicated by the clandestine insertion of thousands of Russian soldiers after the downing of the Malaysian aircraft MH17 in 2014 (Besemeres, 2016; Larrabee et al., 2017). State-sponsored advanced persistent threats (APT). Sophisticated hacking groups or APT groups are now the centerpiece of Russia’s information dominance campaign (Connell & 18 Vogler, 2017). The Russian Military Intelligence Service (GRU), Federal Security Service (FSB), and Foreign Intelligence Service (SVR) are known to use APT groups 28, 29, and CyberBerkut, among a list of other supporting actors and informal proxies to discredit democracy and increase distrust in Western policy (Polyakova & Boyer, 2018). While associations between the Russian security units and APT groups were identified, e.g. GRU and APT 28, also known as Fancy Bear; and associations between the SVR and APT 29, also known as Cozy Bear, Russia will go to great length to obfuscate attribution to their premiere malware designs and data exfiltration methods (Kollars et al., 2018). Cyber activities typically mirror conventional conflict, e.g. Chinese use of smaller, highnumber attacks, while Russia and the United States use advanced technology in more targeted attacks (Geers et al., 2015). Detecting compromise or the intent to compromise an information system is difficult due to the various layers, location, and timing of seemingly innocuous subcomponents of malware. Identification of complete malware packages is inefficient compared to identifying indicators of compromise (IOC), including file components, scripts, and code characterizing malicious activity (Mandiant, 2010). In June 2017, the Russian state-sponsored hacking group Sandworm Team released the NotPetya virus, a destructive suite of malware that infected over 300 companies in Ukraine and was estimated to have deleted information from 10% of the country’s computers (Greenberg, 2018). NonPetya was assessed to be developed by a state-sponsored APT group, largely due to their leverage of leaked United States National Security Agency (NSA) exploits, self-cloning worms, and the lack of financial motivation (Europol, 2018). NotPetya was a targeted attack on Ukrainian accounting software M.E.Doc, but it eventually spread to 64 countries including Russia (Polyakova & Boyer, 2018). According to Greenberg (2018), NotPetya consisted of the 19 NSA’s EternalBlue exploit to attack a Windows protocol vulnerability followed by Mimikatz, which would retrieve users’ passwords in a systems’ Random Access Memory (RAM). This combination allowed hackers to use retrieved passwords from exploited machines to compromise computers that already received the EternalBlue patch (Greenberg, 2018). Critical infrastructure. A 2014 report from The Center for the Study of the Presidency and Congress anticipated Russia would attempt to “exploit power grid vulnerabilities to achieve strategic objectives” (p. 39). In March 2018, the United States Computer Emergency Readiness Team announced Russian hackers infiltrated several private and government-run energy companies and power plants, which resulted in the United Kingdom issuing warnings to hospitals, gas and electric companies, and water treatment facilities (Keating, 2018). This prompted the United States National Cybersecurity and Communications Integration Center (NCCIC) to issue a similar warning using the Lockheed-Martin Cyber Kill Chain model, dividing the attacks into 7 stages and detailing the forensic artifacts found at each. A modified version of the Lockheed-Martin Cyber Kill Chain with general descriptions of the activities occurring at each stage is shown in Figure 2 (adopted Lockheed Martin, 2019). Figure 2: The cyber kill chain (adapted from Lockheed Martin, 2019). Russia is no stranger to restricting energy for political coercion as demonstrated by the halting of European gas supplies through Ukrainian territory during the Winter of 2008-2009 (Verner, Grigas, & Petit, 2019). The Russian military more recently demonstrated energy 20 restriction when it occupied a natural gas distribution center on Strilkove, a small strip of land northeast of the Crimean Peninsula only four days after the Crimean Parliament’s declaration of independence from Ukraine (Kofman et al., 2017). While Russia is the primary supplier of crude oil to Europe, it is unlikely any future conflict between NATO and Russia would permanently affect oil exports due to the ease of redistribution from global markets (Larrabee et al., 2017). Russia generates considerable revenue from natural gas and could apply pressure to smaller, northeastern and central European countries that are highly dependent on this natural resource, but at great loss to the Russian economy (Larrabee et al., 2017). Permanent interruption to electric power in the Baltic States is unlikely as this would include disconnecting Belarus and Kaliningrad and would require substantial investment in infrastructure to redistribute power (Larrabee et al., 2017). Research by Larrabee et al., (2017) on Russian reluctance to use long-term energy coercion is contradicted by more recent analysis from Verner et al., (2019), which suggested Europe’s energy dependency on Russia will only increase as construction of the Nord Stream 1 and 2 gas pipelines and the completion of the Power of Siberia pipeline to China will provide alternative revenue and the option to cease natural gas exports to Europe for up to a year. While Russia actively pursues alternative sources of national revenue the Chairman of the Center for Defense Reforms in Ukraine Oleksandr Danylyuk (2016) attested that Russia’s motivation for the Nord Stream pipelines are political, not economic. Listed priorities from the 2018 Brussels Summit reiterated NATO’s concern of European critical infrastructure vulnerabilities (Kramer et al., 2018). Recent European Council (2018) declarations suggested a long-term strategy to decrease Europe’s energy dependence on Russia, to include the modification of existing storage facilities, the construction of cross-country 21 interconnectors, and collaborative risk assessments. Four years ago, FireEye (2015) reported the vulnerability of Norway’s oil and natural supplies as critical nodes in the European energy grid and the probability Russia would target them as strategic nodes if a conflict arose. This same report also indicated Norway received 47% of targeted malware attacks from APT groups, with a combined 47% occurring in the chemicals, manufacturing, mining and energy, and utilities industries. Nordic countries attempted to diversify energy sources by developing a submarine power line between Lithuania and Sweden, named NordBalt. The NordBalt power cable was commissioned in 2016, but its location in the shallow and dense shipping lanes in the Baltic Sea is prone to interference by the Russian Navy and the construction of Russia’s Nord Stream 2 (Verner et al., 2018). The Russian APT group Sandworm Team is associated with the BlackEnergy family malware, which was used to attack energy and other critical infrastructure nodes for a decade (Vann, 2017). The December 2015 cyber attack on Ukrainian power stations consisted of the following components: sophisticated spear phishing campaigns targeting Ukrainian energy companies, the presence of BlackEnergy 3 at each infected energy company, theft of administrator credentials on each infected network, virtual private network (VPN) use to access industrial control system (ICS) networks, remote access tools, damaged equipment at the firmware level, malware used to erase master boot records and specific systems logs, and DoS attacks on customer support call center (Lee et al., 2016). While BlackEnergy received three significant upgrades, security companies have catalogued hash algorithms like MD5 of specific components of the BlackEnergy payload that can be used to scan existing systems, third party software, and identify the its presence on adversary information systems when exploited (Vann, 2017). Although malware such as BlackEnergy 3 and the KillDisk eraser software were initially 22 accredited for causing the Ukrainian power outages, they merely enabled infiltration and delayed restoration; the outages occurred from the hacker’s direct interaction with the ICS (Lee et al., 2016). A survey at Black Hat Europe (2017) showed 77% of respondents anticipated a cyber attack across European critical infrastructure within 24 months, and 42% listed cyber espionage by nation-states as the greatest threat to Europe’s critical infrastructure. Cyber crime and organized crime. A 2013 study concluded there was insufficient evidence to suggest cyber crime was predominately a function of organized criminal syndicates (Lusthaus, 2013). Broadhurst et al., (2014) expanded research by identifying a positive correlation between the size of a cyber crime group and its financial motivation, noting that statesupported cyber criminal groups demonstrate clear inclination for economic or political targets. This conclusion was seconded when a Black Hat Europe (2017), survey of 127 top European IT and security professionals claimed that cyber attacks from criminal organizations and nationstate supported threat groups are exhausting cyber defenses. In addition, statistics indicated knowledge of zero-day vulnerabilities and support from organized crime or nation-states account for 35% of the ‘most-feared cyber attacker’ profile, preceded only by insider threat (Black Hat Europe, 2017). Russian cyber criminals are known to use spear phishing campaigns to lure target users to compromised websites containing malicious code that disables anti-virus software of victim’s operating system (Geers et al., 2015). In February 2017 Russian-born United States citizen Alexander Tverdokhlebov was arrested for operating a 500,000-node botnet to steal over 40,000 credit card numbers which were used or sold to amass over 10 million dollars (Cimpanu, 2017). A botnet is a network of infected computers that are commanded to perform various functions like retrieve and exchange information, steal personally identifiable information, or send spam (Cimpanu, 2017). 23 Tverdokhlebov established laundering services where money from stolen credit cards numbers and compromised bank accounts would be used to purchase high-value items only to be resold with the profits distributed to Tverdokhlebov, or money would simply be extracted from ATMs (Cimpanu, 2017). The United Nations manual on Criminal Intelligence (2011) asserted a deep connection between organized criminal networks and their operational population. United States officials were warned in late 2008 of the threat Russian networks posed in Central Europe, but United States foreign policy did not change (Conley, Mina, Stafanov, & Vladimirov, 2016). The Commander of the Lithuanian Border Guard claimed the leaders of the criminal organizations capitalizing on Lithuania’s porous border with Russia are citizens and residents of Russia (as cited in Brown, 2017). Although illicit tobacco smuggling in the European Union costs over 10 billion euros in lost tax revenue, the Lithuanian State Security Department is less concerned with illegal trade compared to the Russian FSB efforts to infiltrate the Lithuanian State Border Guard (Brown, 2017). The Spanish Ministry of Housing Development (2018) reported Russian purchase of Spanish property has doubled the last decade, the most popular destinations falling within the autonomous communities of Valencia, Catalonia, and the Canary Islands, and Alicante as the most popular city (as cited in HTBIS, 2019). In fact, the leader of the Carbanak (Cobalt) hacker group was arrested in the Spanish city of Alicante in March 26, 2018 (Cimpanu, 2018b). The Carbanak group executed over 90 hacks in 40 countries accounting for over a billion dollars stolen from sophisticated attacks of banks, ATM networks, and electronic money transfer services (Cimpanu, 2018b). In July 2018, three high-level members of the Carbanak group, all Ukrainian citizens, were arrested in Germany, Poland, and Spain (Ashford, 2018). Kaspersky’s 24 (2015) comprehensive analysis of Carbanak illustrated continued evolution of targeted malware for the financial industry backed by criminal adaptation of sophisticated reconnaissance exploits typically representative of APT groups. Alicante, Spain was also home to Zakhar Kalashov, a Georgian crime boss with connections to Russia’s Federal Security Service (FSB) and the Russian petroleum company Lukoil (Rotella, 2017a). Upon Kalashov’s arrest in 2006, Spanish prosecutor José Grinda Gonzales traveled to London to solicit the help of exiled, ex-KGB agent turned MI6 consultant Alexander Litvinenko to secure a longer prison sentence for Kalashov. Litvinenko agreed to testify against Kalashov but was poisoned by radioactive polonium-210 a week before he was scheduled to testify (Rotella, 2017a). Following Litvinenko’s death, the Spanish authorities turned their attention to Gennady Petrov, a senior member of the Tambovskaya criminal syndicate in St. Petersburg, Russia (Rotella, 2017a). During the investigation on Petrov, a team of international police intercepted hundreds of communications between Petrov, powerful businessmen, recognized criminals, and high-level members of the Russian government, to include the Russian defense minister Anatoly Serdiukov (Rotella, 2017a). Investigators claimed the Russian mob in Spain did not simply bribe local officials but were deeply embedded in legitimate businesses and politics in both Spain and Russia, and laundered money for their mutual benefit (Rotella, 2017a). Petrov was arrested by Spanish officials in Mallorca, Spain in 2008 (Rotella, 2017a). Information operations (IO). Seventy percent of the top 20 countries with the lowest perceived levels of corruption are in Western Europe, with the United States falling at 22 (Transparency International, 2018). In this same survey, Russia occupied position 138 out of 180 possible countries. One component of national corruption is overt and covert disinformation 25 campaigns. Russia overtly uses stated-controlled media companies such as Sputnik and RT to promote pro-Russian narratives while countering international news with false counterpoints, and they covertly finance and direct propaganda factories like the Internet Research Agency to flood social media sites with false information and Russian nationalism (Polyakova & Boyer, 2018). Russian television network RT played a crucial role in Russian IO campaign in Ukraine, receiving a 41% increase to its 300-million-dollar budget (Perry, 2016). RT’s effectiveness in disseminating pro-Russian media and suppressing Western narratives prompted Lithuanian Minister for Foreign Affairs Linas Linkevicus to claim Russia’s propaganda and military machines are equally destructive (as cited in Perry, 2016). Giles (2016) referred to Russia’s use of IO as a) information-psychological warfare, focused on adversary armed forces and population, and b) information-technology warfare, focused on information systems. Giles (2016) noted one of the principle goals of informationpsychological warfare was heavily based on agitation and the creation of a scripted psychology within the adversary that influences action in a predetermined direction. An example of information-psychological operation is the September 11, 2014 Columbia Chemicals hoax where false texts, tweets, web pages, and videos were posted of an imaginary explosion, putting St. Mary Parish, Louisiana in a state of panic for two hours (Chen, 2015). Later in 2014, another hoax reported an outbreak of Ebola in the United States city Atlanta, Georgia (Chen, 2015). Chen (2015) eventually travelled to St. Petersburg Russia to interview Ludmila Savchuk, an employee of the Russian propaganda factory the Internet Research Agency. Savchuk admitted the Internet Research Agency was an instrument for Russian propaganda and consisted of 400 employees working rotating, 12-hr shifts, for salaries equal to Russian university professors. Savchuck posted pro-Russian and anti-NATO/anti- 26 Western content on a variety of Russian and international media and social media sites, with a quota of 150 posts per shift (Chen, 2015). After the United States announced intent to withdraw from the 1987 Intermediate-Range Nuclear Forces (INF) Treaty in October 2018, claiming Russian violation for its development of the 9M729 land-based cruise missile, NATO Secretary General Jens Stoltenberg claimed the Alliance had no intention of initiating an arm race with Russia (Brzozowski, 2019). The INF Treaty is a bilateral agreement between Russia and the United States to prohibit the possession, production, or testing of land-based ballistic missiles with ranges between 500 to 5,500 kilometers with specific intent to reduce arms tensions between Russia and Europe (Hurd & Chachko, 2018). Russia claimed the United States created conditions for the dissolution of the INF Treaty by implementing the Aegis Ashore missile program, in addition to citing the United States use of drones as a treaty violation (Ramsay & Robershaw, 2018). In February 2019, state-owned, Russian television indicated potential nuclear targets within the United States; two of the three were closed military bases (Vesti Nedeli, 2019). This report came days after Russian President Putin claimed Moscow would not avoid a confrontation similar to the 1962 Cuban Missile Crisis (Vesti Nedeli, 2019). During the State of the Nation speech following his re-election in March 2018, Vladimir Putin lauded Russia’s recent conventional military advancements, but these advancements cannot be verified by independent analysts and pose serious technical and financial challenges to Russia even with recent increases in military spending (Ramsay & Robertshaw, 2018). NATO does not officially define IO but outlines the application of IO in military strategy and operations in Allied Joint Doctrine for IO 3.10 (AJP-3.10) (NSO, 2009). NATO military policy for information operations MC 0422/6 is in draft and will likely update and improve NATO’s IO implementation strategy (NSO, 2018b). 27 Subverting democracy. According to a 2018 Washington Post report, Russia interfered in the Dutch EU and Ukraine Agreement (2016), Brexit (2016), the Italian constitution referendum (2017), the French presidential elections (2017), the crisis in Spanish Catalonia (2017), and German parliament elections (2017) (Noack, 2018). There is ample evidence of governments and political parties dedicating considerable funds to utilize social media to propagate political agenda (Bradshaw & Howard, 2017). Russia utilized low-cost and expansive propaganda prior to the invasion of Ukraine to boost support to Russian foreign policy in the area (Raugh, 2016). According to Freedom House’s Freedom on the net report (2017), 18 countries’ elections experienced disinformation campaigns between June 2016 and May 2017. Motivation to disrupt or influence free elections is technology agnostic and while improvements in election technology provided a new medium for interference, digital and analog voting systems are vulnerable unless monitored by an objective oversight system (Past, 2018). During the 2016 presidential elections in the United States, then Federal Bureau of Investigation Director James Comey announced hackers attempted to compromise voting stations, but inconsistencies of voting technology between states impeded their success (as cited in Mansfield-Devine, 2018). Countries with modernized election systems will be more homogeneous or consist of the same hardware and software in different voting locations, making it easier to compromise the voting infrastructure compared to a network system with variations in node composition (Tavakoli, 2018; as cited in Mansfield-Devine, 2018). The six months following the 2016 United States Presidential election saw massive increases in social media posts by the Russian troll-farm Internet Research Agency to include: a) 45% increase in Facebook ads, 59% increase in Facebook posts, 238% in Instagram posts, and 52% in Twitter posts (Howard, Ganesh, & Liotsiou, 2018). 28 Russia was one of six countries that took steps to reduce independent media while declaring the state was the only trusted voice in the media (Shahbaz, 2018). While the impact of election interference is still fresh on the global consciousness, recent research demonstrated a negative effect of restrictive use of the Internet. Social media and fake news’ reduced efficacy to influence elections is apparent by the decline in Internet freedom in 2018. Shahbaz (2018) reported that half of the countries whose Internet freedom decreased in 2018 positively correlated to election interference. In October 2017, the Autonomous Community of Cataluña held a referendum for separation of from Spain. Weeks prior and immediately following there was a barrage of proCataluña and anti-Spain propaganda on social media (Alandete, 2017). Analysis of this content showed almost 50% were from pro-Communist supporters in Venezuela, half of which were anonymous accounts run by a botnet and used to spread Russian media previously released on Russian news organizations RT and Sputnik (Alandete, 2017). Although Cataluña’s independence election was declared unconstitutional, it struck fear in businesses headquartered in Barcelona and other Catalan cities resulting in the relocation of over 3000 businesses, at a loss of 11 billion euros to Cataluña (The Economy Journal, 2019). According to Rotella (2017b), the Russian mob was lobbying for Russian businesses by capitalizing on political and jurisdictional differences between Cataluña and Spain years prior to the 2017 referendum for independence. Recent research suggested a combined government-industry effort to develop computational counter-propaganda systems able to detect and mitigate the negative societal effects of misinformation (Polyakova & Boyer, 2018). In addition, NATO and the European Union could establish relationships with social media companies to leverage existing and emerging industry technologies that can provide early warning of false information aimed at 29 undermining democracy (Polyakova & Poyer, 2018). The Prague Declaration (European Values, 2017) provided suggestions for Western democracies to counter Russian disinformation campaigns: a) acknowledge the threat of active Russian propaganda and intentional influence in democratic elections while establish punitive measures for interference, b) investigate and expose Russian propaganda, especially in demographics most prone to persuasion, c) bolster national and international strategic communications organizations promote their cooperation, d) establish working groups that cross national borders to explain the Russian hybrid threat to nations not directly affected, and e) increase information sharing and training outside the expert community to raise Europe’s baseline knowledge of the impact of Russia’s hostile influence. Electronic warfare. Senior fellow at the Center for Transatlantic Relations Dr. Hans Binnendijk outlined Russia’s increasing ability to develop, deploy, and utilize anti-access/area denial systems to reduce NATO air-superiority (2016). Russia demonstrated competent electronic warfare capabilities while disrupting Ukrainian border communications prior to attacks on headquarters facilities (Roblin, 2018). Russian Spetsnaz forces were reported to use Ratnik, man-portable geolocation computers to transmit Ukrainian forces’ positions to increase the accuracy of artillery (Cranny-Evans et al., 2018). Roblin (2018) noted that while Russian area-denial and electronic warfare systems may be effectively employed, their low numbers force Russia to position and relocate according to critical area priority. The Union of Concerned Scientist Satellite Database showed the United States as the global leader in operational satellites with 830; 385 reserved for government use (Statista, 2019). All of Russian satellites in orbit account for only 38% of those managed by the United States Government. Considering the level of United States dependence on global navigation satellite systems (GNSS) it is feasible Russia could interfere with one or both of the dominant GNSSs in 30 Europe, Global Positioning System (GPS) or Galileo by jamming, or barrage interference of the electromagnetic spectrum on uplink/downlink frequencies, or spoofing, the generation of a false GNSS signal to falsify position, navigation, and time data (Heue, 2018). While GNSS data is largely referenced to navigation, it is imperative to note the importance of GNSS timing data to synchronize precision instruments used in the energy sector (Heue, 2018). Russian GPS jamming increased in 2017 reaching its peak during NATO’s Trident Juncture exercise from October 16 to November 7, 2018 (Nilsen, 2019). The Norwegian and Finnish governments reported interference with military and civilian navigation systems during Trident Juncture and Norway received Russian officials to show proof of GNSS jamming, but Russia claimed the allegations were preposterous (BBC News, 2018; Nilsen, 2019). The ability to jam GNSS is not new and NATO developed a device capable of geolocating a GNSS jammer through direction of arrival (DOA) of a jammer’s signal (Stolk, 2016). Geolocation through DOA occurs as several lines of bearing are made from a device emitting an electromagnetic signal as it travels to a sensor (Hale, 2012). Geolocation through DOA is maximized with a) a stationary antenna array and a mobile target or b) a mobile antenna and a stationary target (Hale, 2012). With Russia’s recent use of GNSS jammers on GPS frequencies, it should be noted that a portion of the Russian-owned GLONASS GNSS frequency spectrum partially overlaps the GPS L2 band (LabSat, 2019; Novatel, 2019). Research indicated highly sensitive, single frequency GPS receivers mitigate a jammer’s effectiveness by reducing the quantity of the jamming signal, thus lowering the receiver’s signal to noise ratio (Borio, O’Driscoll, & Fortuny, 2013). The Russian Army used the RB-301B Borisoglebsk-2 to both jam and collect cellular phone transmissions and reportedly used this device to intercept Ukrainian forces using mobile 31 phones in attempt to correct artillery file (Cranny-Evans et al., 2018). Additionally, Russian troops used the Leer-3 direction finding equipment to fix Ukrainian forces mobile phones by identifying their GPS coordinates (Cranny-Evans, Cazalet, & Foss, 2018). Tracking cellular phones has been documented as early as 1996 when such a device was used to catch the notorious hacker Kevin Mitnik (Naarttiärvi, 2016). The common name for these devices is IMSIcatchers, which stands for International Mobile Subscriber Identity, or the 15-digit number used by Global System for Mobile Communications (GSM) and Universal Mobile Telecommunications System (UMTS) cellular telephone networks to identify a subscriber (Pell & Soghoian, 2014). Passive IMSI-catchers act as receivers collecting cell phone metadata as a device comes within range, while active IMSI-catchers emulate base transceiver stations (BTS), lock the device in a traffic channel, and interrogate the signal collecting valuable metadata while allowing it to be geolocated by additional hardware (Naarttiärvi, 2016). Some IMSI-catchers are able to break weaker forms of encryption algorithms (Lilly, 2017). It is possible to actively jam 3G or 4G networks to force subscribers onto weaker telecommunications architecture to facilitate collection and decryption (Lilly, 2017). United States Naval War College academics Kollars and Petersen (2018) argued that effort should be spent on understanding technical IO, to include their structure, tools, and strategic goals, prior to countering the psychological aspect of IO, which are performed at lower levels and less likely to reveal Russian priorities. Janda (2018) suggested Western leaders must understand that a) it is imperative to increase Europe’s counter-disinformation campaign efforts and research Russia’s targeted demographic for propaganda, b) publicizing Russian disinformation with highly correlative evidence is the best weapon against IO, and c) ambiguous punitive consequences invite an escalation of tensions Giles (2016). 32 Compromised software and hardware. Industry leading cybersecurity firm FireEye observed a nation-state develop sophisticated malware only to sell it on the black market after implementing counter-malware defenses (Geers et al., 2015). In 2017, the Lithuanian Defense Vice-Minister Edvinas Kerza announced the identification of compromised Russian-made software on several state-owned information systems (The Baltic Course, 2017). Pro-Russian trolls have posted contradictory information in the comments section of Latvian blogs then directing members to click on infected links for further information (NATO Strategic Communications Centre of the excellence, 2016). In March 2018, the microprocessor company AMD publicly announced manufacturerinstalled backdoor vulnerabilities in a series of its chipsets allowing access through known malware like MasterKey, Fallout, RyzenFall, and Chimera (Cimpanu, 2018a). These exploits are second-stage vulnerabilities that allow manipulation once an adversary establishes administrative privileges through other means (Gillware, 2018). These vulnerabilities are similar to Spectre and Meltdown, which are the names given to a series of processor vulnerabilities within nearly all computer processors manufactured in the last 20 years (Garcia, 2018). The Spectre and Meltdown vulnerabilities exist to maximize efficiency of data processing and patches to these vulnerabilities result in noticeable reductions in performance (Newman, 2019). NATO’s Approach to Current & Future Challenges While official relationships with non-NATO hybrid warfare and cybersecurity entities were not directly declared in the Brussels Summit, the nature of these threats and the push for whole-of-government approaches suggests NATO will form close relationships between NATO Headquarters, Allied Command Operations, the NSHQ, the NATO Communications and Information Agency (NCIA), the Organization for Security and Cooperation in Europe (OSCE), 33 the NATO CCDCOE, Europol, the United States Cyber Command, and the European Centre of Excellence for Countering Hybrid Threats (NATO, 2018e). Although NATO could benefit from cross-organization collaboration, establishing official relationships do not directly equate to improvements in dialogue or policy. Besemeres (2016) noted the lack of participation by the United States and United Kingdom in the OSCE’s Contact Group on Ukraine, and Russia’s persistent use of veto power within the OSCE suspended Western involvement and resulted in favorable reports and indirect OSCE support to Russian priorities. A 2016 special issue of the Combating Terrorism Exchange (CTX) magazine, was devoted to Countering Hybrid Warfare: The Best Uses of SOF in a Pre-Article V Scenario and included topics related to NSOF integration with conventional forces (CTX, 2016). At a 2018 Assembly of the NATO Sub-Committee on Future Security and Defence Capabilities, Moon (2018) presented rationale supporting NSOF’s utility to counter gray zone tactics through special reconnaissance, intelligence collection and targeted operations. Cyber reconnaissance and opensource intelligence are inexpensive compared to traditional forms and requires a mastery of freeware tools available on the Internet (Taylor, 2017). The NATO Centre of Excellence on Strategic Communications (STRATCOM) (2016) advised that adversaries will continue to exploit cyberspace and social media to control narratives, influence psychology, and distribute malware, and these strategies will adapt in accordance with changes in the information environment and communication habits. Giles (2016) reiterated that while some sarcastically implied Russian SOF in Crimea operated with iPads, the synergies between Russian hybrid warfare tactics to include cyber operations is closer to the truth than fiction. Unfortunately, one of the greatest challenges facing European strategy for countering 21st century threats is the prioritization of individual national threats (SERA, 2016). While the 34 permanent presence of NATO forces is politically sensitive, the United States would pursue bilateral deterrence agreements with European countries if forces are required and a NATO consensus could not be met (Kramer et al., 2018). Committee number 5 of the 2016 Session Europeenne des Responsables D’Armement (SERA) found that national priorities to counter hybrid threats varied across five categories: cyber, criminal activity, terrorism, migration, and hostile border countries. These differences are influenced by geography, history, political strategy, and perceived roles within NATO (SERA, 2016). Figure 3 illustrates a hypothetic example of differing national threat perceptions contributing to the challenges of a consolidated and comprehensive European or NATO counter-hybrid threat strategy. Figure 3. Differing national levels of threat perception (SERA, 2016, p. 115). Hybrid warfare strategy. In 2016, Senior Fellow at the Center for European Policy Analysis Janusz Bugajski recommend permanent force presence, increased infrastructure of early 35 warning systems, improved capabilities to counter hybrid threats and most importantly, revision of Article 5 definitions to account for cyber warfare and other acts of subversion. In the same year, alternative strategies were described. European security experts Glatz and Zapfe (2016) suggested qualitative improvement in NATO nations’ militaries would be a greater deterrent than deployed forces. Others countered this logic suggesting an increased NATO military footprint in countries like the Baltics and Poland is not a direct counter to the last decade of Russian aggression, but an expression of insufficiently implemented NATO military strategies (Kiesewetter & Zielke, 2016). Raugh (2016) emphasized the economic cost of combatting hybrid threats, noting the lack of inexpensive solutions suggested policy makers distribute limited resources to a well-equipped, focused counterforce (Raugh, 2016). This advice for asset allocation was imperative while NATO defense spending remained a sensitive topic with the United States providing 70% compared to Europe’s 25% contribution, although Alliance nations bordering Russia have recently increased their contributions (Bugajski, 2016). In December 2016, the United States Army’s Asymmetric Warfare Group (AWG), published the ‘Russian New Generation Warfare Handbook’, intended to highlight atypical threats outside the purview of traditional American understanding of Russian military doctrine and tactics (Trevithick, 2017). A senior Russian Defense Committee official described the AWG’s handbook as irrelevant, claiming Americans “are unable to fight the Russians in military and technical terms or in moral and psychological training” (Klintsevich, 2017, para 7). Recent multi-national research described hybrid warfare as the asymmetric application of several power instruments in three dimensions with attention on creativity, cognition, and ambiguity––all aspects of Russia’s chosen hybrid applications over the last decade (Cullen & ReichbornKjennerud, 2017). 36 NATO’s counter-hybrid warfare narrative was mentioned in the 2014 Wales Summit, it expanded in the 2016 Warsaw Summit, and was a predominant theme in the 2018 Brussels Summit. NATO’s stance on hybrid threats was outlined in the Brussels Summit declaration (NATO, 2018e). NATO is prepared to aid Allies in all stages of hybrid conflict and could invoke Article 5 in the event of armed attack, and NATO will establish Counter Hybrid Support Teams (CHST) to assist Allies preparing for and engaged with hybrid threats. Shortly following the Brussels Summit Kramer et al., (2018) suggested NATO could confront hybrid challenges by thoroughly resourcing CHSTs, establishing and coordinating CHST, NSOF, and other national and European counter-hybrid capabilities through a single command-and-control entity, and develop an indications and warning system for hybrid threats. In 2016, The Joint Intelligence and Security Division (JISD) was established to monitor the increasingly ambiguous lines between military and civilian activities and between peace and war (Freytag von Loringhoven, 2017). The JISD expanded its role to include analysis of hybrid threats, cyber threats, and terrorism and to maintain close connectivity to the greater NATO intelligence apparatus and the NIFC (Freytag von Loringhoven, 2017). NATO supported the establishment of the European Centre of Excellence for Countering Hybrid Threats (Hybrid COE) in 2017, but also maintained a NATO Hybrid Analysis Branch (Kramer et al., 2018). The Hybrid COE, NATO Hybrid Analysis Branch, and the European Union Hybrid Fusion Cell collaborate through workshops, exercises, and video teleconferences to increase situational awareness of hybrid threats (NATO, 2018d). While the Hybrid COE does not publish policy, doctrine, or serve an operational capacity, it acts as a platform for strategic discussions, research, capabilities enhancement on hybrid matters (Hybrid COE, 2019). 37 To exercise collective self-defense against an adversary employing hybrid warfare, NATO conducted Trident Juncture 2018 under the synopsis of an Article 5 invasion of Norway (Masters, 2018). Among several experiments performed during Trident Juncture was a) an autonomous military logistics capability and b) a capability to assess the information environment, to include using new technology to monitor NATO’s and adversary communications, filter open-source information regarding the exercise, and synthesize the strategic to tactical relevance of information along a continuum (Paxton, 2018). A fusion cell then collated and analyzed data from the information environment in the context of the exercise commander’s information requirements (Paxton, 2018). Cyber strategy. Soviet Union historian Anne Applebaum (2018b) suggested NATO reframe itself as the epicenter of Western cyber and information security. The Tallinn Manual was updated to version 2.0 in 2017. The update was not to serve as best practices or doctrine, but to describe international law governing cyber warfare (Schmitt, 2017). NATO Secretary-General Jens Stoltenberg (2018) confirmed cyber attacks can qualify as a breach of Article 5, but the exact threshold for triggering such events would not be communicated to deny strategic information to NATO adversaries (as cited in Gramer, 2018). An optimal cyber defense strategy is one where timely and relevant cyber threat information feeds a dynamic cycle directing modification of an organization’s nodes to reduce internal vulnerability. NATO’s stance on cybersecurity was outlined in the Brussels Summit (NATO, 2018e) declaration with highlights announcing the creation of a Cyberspace Operations Centre in Belgium and clarifying that cyber attack attribution is a sovereign state responsibility (Kramer et al., 2018). Nordic Council President Michael Tetzschner (2018) suggested increased cooperation between the Nordic Council, the United States, and the European Union, and 38 inclusion of Baltic countries Estonia, Latvia, and Lithuania in co-operative cyber defense efforts. While not directly stated in the Brussels Summit declaration (2018e) the battalion-sized NATO battlegroups in Lithuania, Poland, Estonia, and Latvia suggests these elements could be leveraged to facilitate the sharing of cyber-related information. The NCIA is responsible for the preponderance of NATO communications infrastructure to include C2 for active Article 5 operations (NCIA, 2019a). The NCIA advertised its Cyber Security Service Line as: providing specialist cyber security-related services covering the spectrum of scientific, technical, acquisition, operations, maintenance, and sustainment support, throughout the life cycle of NATO information communications and technology, enabling secure conduct of the Alliance’s operations and business in the NNEC environment and in the context of NATO’s Command, Control, Communications, Computers, Intelligence, Surveillance. (C4ISR). (NCIA, 2019b, para 2) The NCIA participates in NATO-led innovation through a continuum of interoperability events consisting of a) the Think-Tank for Information and Decision Execution Superiority (TIDE) Sprint, b) the Coalition Warrior eXploration, eXperimentation, eXploration, eXercise (CWIX), and c) the TIDE Hackathon (Allied Command Transformation, 2019). These events combine academia, industry, military strategists, and government technology professionals to identify and resolve interoperability challenges and promote collective defense technology (Allied Command Transformation, 2019). The stated objectives of the 2018 TIDE Hackathon were to develop architectural models and strategies to address business problems, develop software and hardware to facilitate business solutions, and promote interoperability through information sharing and education of NATO Enterprise Architecture stakeholders (Allied 39 Command Transformation, 2018). Commercial-government partnerships like that between the European Commission and European Cybersecurity Organization (ESCO) is an example of forward progress to take advantage of sophisticated data analysis of private companies (Limnéll, 2018). Combining military and non-military specialties for technology enhancement mirrored Gerasimov’s 2013 comments on the future of Russian warfare, which indicated the importance of interagency cooperation, including local and federal LE, state institutions, and the military (2013). The development of secure communications technologies by nations and their militaries will not outpace civilian mobile networks and the global desire for connectivity and higher bandwidths (Ciufo, 2018). Reliance on networked information systems to command machines or produce intelligence implies reduced precision as systems exposed to additional nodes are also exposed to their implicit vulnerabilities (Libicki, 2012). While accessing networked systems through the Internet is economical, external access to networked systems can occur through illicit access by a foreign agent, compromise of vulnerable logic circuits to misrepresent data, or the injection of alternative communications via a rogue radio frequency (Libicki, 2012). The NATO CCDCOE runs two major cybersecurity exercises per year, Crossed Swords and Locked Shields (Latvian Public Broadcasting, 2018). Crossed Swords brings experts in cyber defense, penetration testing, digital forensics, and situational awareness to train and operate the latest technology and techniques to compromise information systems (CCDCOE, 2019b). Those trained at Crossed Swords play the red team during the Locked Shields exercise where several blue teams from NATO nations and international organizations attempt to protect a fictitious country as it experiences a series of complex cyber attacks on electrical grids, safety 40 networks, communications networks, and other hybrid activities like the use of drones (CCDCOE, 2019c). Rubio Melón, Väisänen, & Pihelgas (2018) tested a data aggregator (named ADAM) and data visualizer (named EVE) that defines and displays cybersecurity alerts during Locked Shields 2017. Rubio et al., (2018) described accessing EVE through external aggregators through “HTTP POST request to an EVE’s web service” (p. 10-6). Referring to cyberspace as a traditional warfighting domain (land, sea, air, and space) connotes the application of symmetric counter-strategies executed on the premise of control within a physical medium. Unfortunately, cyberspace is a malleable medium comprised of asynchronous connections and nodes, of which a user cannot exercise absolute control (Libicki, 2012). United States Marine Corps University professors and Cato Institute researchers Valeriano & Jensen (2019) argued the validity of past offensive cyber operations and their strategic effects, suggesting the United States’ new cyber strategy is one which “increasingly sees preemption as the only viable path to security” (p. 3). Valeriano & Jensen (2019) concluded that cyber attacks between 2000 to 2016 did not promote or deter subsequent attacks and most cyberspace activity consisted of political warfare. However, while offensive cyber operations did not correlate to escalatory cyber retaliation, they are less effective than cyber restraint because offensive cyber actions increase the general risk of escalation while cyber restraint combines proportional, “active defense and coercive diplomacy” (Valeriano & Jensen, p. 5). Tactical to strategic cyber intelligence requirements. A Commander’s Critical Information Requirements (CCIRs) guide intelligence collection, analysis, and current and future operations planning to ensure a commander has sufficient, relevant, and timely information to make decisions (Pozderka, 2018). There is a direct relationship between CCIRs and the 41 operational spectrum, i.e. strategic CCIRs influence operational CCIRs, which influence tactical CCIRs (Pozderka, 2018). Requirements in NATO are managed by the Collection Coordination and Intelligence Requirements Management Cell (CCIRM) which is responsible to structure requirements based on incoming reports from sub-cells representing the combatant commands, i.e. AIRCOM, LANDCOM, MARCOM (Korkish, 2010). The need for credible, reliable, and actionable cyber threat intelligence is a critical requirement for collective defense against cyber attacks (Schouse, 2015). According to Co-Chair of the DSB Task Force on Cyber Deterrence James Miller Jr., a productive threat intelligence sharing cycle requires close relationships between the United States and its Allies, private sector support, the accepted reality that systems’ vulnerabilities will persist, and a strategy favoring resilience and response speed over information security (2018; as cited in Carlin et al, 2018). The United States Intelligence and National Security Alliance (INSA) noted no absolute demarcation between tactical, operational, and strategic-level cyber intelligence and emphasized cyber intelligence as an analytic discipline that collates information from existing intelligence sources pertaining to the cyber domain (see Figure 4). Figure 4. Cyber intelligence: Responsibilities and interrelations (INSA, 2013, p. 5). Strategic-level cyber intelligence is timely, coherent, and concise non-technical information for senior leaders that draws direct connections between the severity of a 42 vulnerability, the maliciousness of an attack, and their relevance to an organizations’ goals (INSA, 2014a). Strategic cyber intelligence requirements are those CCIRs needed for leaders to make strategic decisions, they are prioritized, and they include assessments of risk, cost, and legal liabilities (INSA, 2014a). Strategic cyber intelligence is analyzed and presented at a macrolevel, often by categorizing vulnerabilities as “people, process, and technology” (INSA, 2014a, p. 9). Operational-level cyber intelligence focuses on the ongoing protection of an organizations business processes and required infrastructure with emphasis on balancing security with operational freedom and productivity (INSA, 2014b). Examples of operational-level cyber intelligence is monitoring the Internet to identify emerging attack vectors, then developing or integrating countermeasures to prevent compromise or, analysis of one’s organization relative to previously compromised organizations with similar profiles and fortifying the information architecture in anticipation for comparable attacks (INSA, 2014b). Operational cyber intelligence facilitates resource allocation and is preceded by matrix analysis of existing and anticipated threats against known organizational vulnerabilities while understanding complete protection is an illusion (INSA, 2014b). Tactical-level cyber intelligence provides the technical where, what, and how of network compromises with enough speed and efficiency to reduce the effects of a cyber attack (INSA, 2015). Tactical cyber intelligence consists of analysis of the cyber environment, an established information lifecycle, collaboration and sharing of tactical cyber intelligence, and cyber data feeds (INSA, 2015). Intelligence Preparation of the Cyber Environment (IPCE) is necessary to provide context to cyber intelligence and includes the following four steps: “define the technical operating environment, describe the impact of the technical operating environment, evaluate the 43 adversary, and determine adversary courses of action” (INSA, 2015, p. 5). Cyber threat intelligence can be collected through non-cyber means such as HUMINT, signals intelligence (SIGINT), and geospatial intelligence (GEOINT) (INSA, 2015). All-source intelligence fusion occurs conjunction with various operational and intelligence cycles where cyber intelligence provides context to threat actors and their platforms (INSA, 2015). The cyber intelligence lifecycle consists of seven steps and guides an organization from the collection of artifacts from compromised assets to the analysis and reporting of the compromise’s impact (see Table 2). Table 2 Cyber Intelligence Lifecycle and Descriptions Step 1 Step 2 Requirements Collection The organizations' cyber needs based on Aggregating data business model, from multiple sources industry, size, and to a customized threat known information database. vulnerabilities. Step 3 Process and Exploitation Step 4 Analysis and Production Step 5 Step 6 Step 7 Dissemination Consumption Feedback Translating data to actionable, repeatable The quality control Consolidation of Customer receipt and processes. Gathering and finalization of an Quality assurance that threat intelligence use of threat indicators of information package intelligence product into consumable intelligence and compromise, e.g. IP to balance technical satisfied customer package for a specific subsequent actions to addresses, file names, and non-technical requirements. audience. mitigate threat. and cryptographic content. hash functions. Note: Cyber intelligence lifecycle and descriptions (INSA, 2015, p. 7). The NCCIC is the United States hub of cybersecurity information sharing between government entities and private sector companies and includes five prerequisites for efficient and effective sharing of threat information: a) a universal data taxonomy and vocabulary, b) a mechanism to receive, store, search and analyze data, c) the ability to automate data sharing, d) the opportunity to include human analysis to machine-parsed data, and e) a data classification scheme (INSA, 2015). The use of tactical cyber intelligence in the above parameters does not preclude traditional information assurance and systems administration approaches to securing an organizations’ network but is a complementary asset to strengthen a networks’ defenses (INSA, 2015). 44 The functions behind cyber war and cyber reconnaissance are different where the former requires authority, access to advanced tools, and technical acuity, and the latter requires the mastery of open-source intelligence (Taylor, 2017). Open-source intelligence is just one component of a series of time-consuming human activities required to plan and execute cyber attacks (INSA, 2013). United States Naval Postgraduate School Professor John Arquilla suggested identifying cyber attack perpetrators poses the greatest challenge to retaliation (as cited in Geers et al., 2015). Proper attribution is a collaborative task requiring divided labor across several specialties and includes technical collection, detailed analysis and investigation, and comprehensive reporting of the incident to support a compelling legal case (Rid & Buchanan, 2015). A panel of senior current and former United States security officers suggested attribution between cyber activity to nation-states, non-state actors, or criminal groups is possible with the proper resources (Carlin, Ledgett, Miller Jr., & Lewis, 2018). Europol (2018) noted that increased public reporting of financially motivated cyber attacks contributed to greater attribution of nation-state sponsorship of cyber attacks on a global scale. United States Naval War College Professor Michael N. Schmitt (as cited in Geers et. al, 2015) suggested, “attribution determinations made without sensitivity to the geopolitical surroundings are seldom reasonable” (p. 4). International military educational institutions must produce cyber leaders with a thorough understanding of the applications of cyber warfare within their operational context (Spidalieri & McArdle, 2016). This is consistent with Berg-Knutsen’s (2016) description of an optimal hybrid warrior as a versatile generalist. Similarly, San Pietro et al., (2018) found specializations in forensics education contribute to a lack of evidentiary significance. Cyber leaders should be able to direct task integration but have the knowledge and skills to participate in critical 45 circumstances (Pearson, 2018). The Senior Research Fellow for Cyber Policy at the United States Institute for National Strategic Studies, Dr. Glenn Alexander Crowther (2017) recommended the careful allocation of military resources in the cyber domain because while cyber will permeate military operations in the future, few individuals will be performing the direct offensive and defensive measures typical of cyber warfare. Technical exploitation strategy. In 2011, the United States established the DoD Forensic Enterprise directing forensics integration into all military operations where applicable (DoD, 2011). This directive listed the Secretary of the Army the Executive Agent for forensics and the Secretary of the Air Force the Executive Agent for digital and multimedia forensics (DoD, 2011). The Defense Forensics and Biometrics Agency (DFBA) handles training and operations using biometrics and traditional laboratory forensics while the National Media Exploitation Center (NMEC) examines documents and digital media (DFBA, 2019). At the time of this research, there was no less than seven organizations affiliated with digital forensics or technical exploitation between NATO and its partner organizations (NSHQ, 2019a; NATO Stability Policing Centre of Excellence, 2016; NATO Counter Improvised Explosive Devices Centre of Excellence, 2019c; European Defence Agency, 2014c; CCDCOE, 2019d; NCIA, 2019c; Digital Forensic Research Lab, 2019). While minor overlap in exploitation training and activities exist, these organizations have different aims, target audiences, and applications. The subsequent paragraphs briefly describe digital forensics and technical exploitation within these organizations without a comprehensive review of their history or current operations. While most of the organizations highlighted offer training in technical exploitation, their course iterations and student throughput is not discussed. 46 As previously stated, NATO doctrine on technical exploitation is outlined Technical Exploitation (NSO, 2015). Technical exploitation is not the responsibility of any one particular organization, but instead, technical exploitation can occur as a part of various activities along the spectrum of conflict. Digital forensics, often called media exploitation or MEDEX, is a subcategory of technical exploitation whose importance increases with advances in digital society (Morris, 2016). Since 2009, the NSHQ has delivered several exploitation courses with varying degrees of digital forensics accounting for no less than 25 days of training per year (NSHQ, 2019a). The digital forensics within NSHQ’s exploitation courses are focused to facilitate the collection, preservation, extraction, and reporting of digital artifacts with an emphasis on triage (NSHQ, 2019a). The NATO Stability Policing Centre of Excellence (SP COE) (2016) suggested that stability policing units should provide forensic services to a variety of military activities including TEO, force protection, and the support to the intelligence cycle. The SPCOE (2016) describes three analogous forensic exploitation systems between itself, counter improvised explosive device (C-IED), and technical exploitation, with each divided into three levels: Level 1 (field exploitation), Level 2 (theatre exploitation), and Level 3 (out of theatre exploitation) (SPCOE, 2016). According to the SPCOE (2016), these three exploitation levels are nearly identical between a) Allied Joint Doctrine for Countering Improvised Explosive Devices (AJP3.15), b) AIntP-10 on Technical Exploitation, and c) the SPCOE’s Stability Policing Framework Concept for Forensics in NATO Stabilization and Reconstruction Operations. The SPCOE (2016) referred to Forensic and Biometric Intelligence as FABINT, a term that combines two of the three disciplines of exploitation that is not found in NATO exploitation doctrine. While the SPCOE delivers a 3-day course titled Preserving a Crime Scene in NATO 47 Operations, it is unknown if digital forensics is a component of this course, yet the SPCOE clearly identifies digital forensics as a key forensic capability in stability policing missions (SPCOE, 2016). The lack of police units’ ability to manage cyber crime cases was studied by Brown (2016) and attributed the lack of technical resources, funding, and training. While many exploitation activities are performed by LE units within Alliance countries, they receive lower priority to training basic staff functions (NATO Military Police Centre of Excellence, 2019). NATO’s Counter Improvised Explosive Devices (IED) Centre of Excellence (C-IED COE) mission is to support the Alliance’s counter-IED effort through information sharing, technology development, and training (C-IED COE, 2019c). The C-IED COE provides a 2-week weapons intelligence team (WIT) course focused on training the response and investigation of IED events and the production of technical, forensic intelligence reports (Level I) (C-IED COE, 2019b). Additional C-IED COE courses are Basic Field Exploitation Course, a DOMEX awareness course that includes digital device extraction (Level II), and two attack the network courses focused on network analysis. As previously stated, NATO doctrine for C-IED is found in Allied Joint Doctrine for C-IED (AJP-3.15) (NSO, 2018c). In 2011, the European Defence Agency (2014b) deployed the multinational counter-IED exploitation laboratory (MNTEL) Afghanistan, which examined over 6 thousand IEDs over its three-year tour. In 2013 equipment from the MNTEL was redeployed to the Netherlands to establish the Joint Deployable Exploitation and Analysis Laboratory (JDEAL), focused on providing facilities, training, and research and development for full-scope improvised explosive device (IED) exploitation (European Defence Agency, 2014c). According to the European Defence Agency (2014a), the JDEAL will develop two expeditionary exploitation laboratories to support member nations. The number of training iterations and the specificity of JDEAL’s 48 training are unknown, but likely includes Level II activities that cannot be conducted in the field due to either time, physical security, or the technical complexity and sensitivity of the forensic task. Digital forensics training also occurs at the NATO CCDCOE and the NCIA. The NATO CCDCOE forensics training includes an Introductory Digital Forensics Course, a Smartphone Security and Forensics Course, and a web-based Digital Forensics and Digital Evidence course (CCDCOE, 2019d). The CCDCOE’s forensics courses are aimed at systems administrators and other network operations staff who may be required to conduct investigations into system and network compromises (CCDCOE, 2019d). The CCDCOE also delivers highly complex courses like malware analysis, systems defense, and ICS defense (CCDCOE, 2019d). The NCIA offers two courses focused on introduction to digital forensics that total 8 days of training (NCIA, 2019c). Although the NCIA does not deliver many forensics courses, its training catalogue is extensive and includes dozens of advanced cybersecurity training and computer courses covering wide topics from leading cyber organizations, cyber intelligence, firewall management, database management, and networking and infrastructure development (NCIA, 2019c). The target audience for most NCIA courses is technical professionals and leaders of technical offices within organizations, and the courses focus on office-based activities, not tactical-level activities that would occur within a conflict zone. In addition, there is strict criteria for who attends NCIA courses, whereby there is an expectation that students’ current or future assignments are directly related to the training (NCIA, 2019c). The Atlantic Council is a non-profit think tank focused on shaping international policy in through detailed analysis of challenges facing the Atlantic Community (Atlantic Council, 2019). Research by the Atlantic Council helped popularize the importance of digital artifacts created 49 from social media when a Russian soldier posted a geo-tagged picture of himself within Ukraine (Czuperski et al., 2015). The Atlantic Council operates the Digital Forensic Research Lab whose mission is to promote objective truth to expose disinformation through open source research (DFRLab, 2019). The DFRLab researched Russian disinformation in Syria, Venezuela, and India and published findings on the online publishing platform Medium.com (Medium, 2019). NATO SOF & NSHQ The 2006 Riga Summit launched the NATO SOF Transformation Initiative, which established the Director of Special Operations, the NATO SOF Coordination Centre (NSCC), and the NATO Special Operations School (NSOS) (Moon, 2018). The NSCC was rebranded in 2010 as the NSHQ with the responsibility to coordinate the development, training, and interoperability of NSOF (Dorschner & White, 2014). The NSHQ quickly developed organic intelligence and exploitation capabilities to include training and education courses focused on NSOF requirements in these and other operational disciplines (Ara, Brand, & Larssen, 2011). The NSOS currently delivers seven intelligence and exploitation courses covering on-target biometrics, forensics, and DOMEX, to advanced forensics courses covering digital media, cell phone, and drone exploitation, and analytic courses teaching the fundamentals of intelligence supporting NSOF missions and human network analysis (NSHQ, 2019a). As previously stated, NATO has made headway developing standards for biometrics, but this progress does not include traditional laboratory forensics or digital forensics (Lunan et al, 2018; NSO, 2013a; NSO, 2016). It is common to hear the following phrase within the NSOF community: SOF conducts tactical missions which produce intelligence that generates strategic effects (Krott, Livingston, & Morales, 2012). To enhance intelligence support to the Commander of NSHQ and to NSOF 50 operations in the International Security and Assistance Force (ISAF) in Afghanistan, two intelligence units were created, the Special Operations Intelligence Branch (SOIB) and the Special Operations Forces Fusion Cell (SOFFC), respectively (Ara et al., 2011). Dorschner and White (2014) noted a remarkable level of intelligence sharing, all-source intelligence fusion, and cross-domain analysis in the SOFFC. Vores (2012) observed that military and civilian personnel who attended NSOS courses had noted advantages integrating in the SOFFC, using exploitation from the battlefield, and processing coherent intelligence products for SOF mission planning. The principal tasks of NSOF are Military Assistance (MA), Special Reconnaissance (SR), and Direct Action (DA) (Moon, 2018). The nature of SOF missions position them as a premiere resource to execute pre-conflict operations to establish control of potential conflict zones, develop partner forces tactics, techniques, and procedures, and conduct targeted strikes in denied areas (Vores, 2012). Precision targeting by NSOF can reveal hybrid networks for exploitation and can deter Russia by increasing attribution to gray zone activities and the political risk of directing them (James, 2016). An official from the Norwegian Defence Research Establishment described the ideal counter-hybrid warfare operator as an educated, adaptable, versatile generalist capable of integrating with any group, a “grand strategy enabler” (BergKnutsen, 2016). SOF’s use in peacetime is ideal due their ability to establish connections quickly with disparate groups, their ability to work autonomously utilizing streamlined decision cycles, and their ability to perform a variety of complex tasks with high proficiency (Shamir & Ben-Ari, 2016). Subsequent research by Breede (2018) agreed with Shamir and Ben-Ari’s (2016) framework and suggested SOF capabilities during peacetime should focus on military assistance activities. SOF are a scarce resource in an era of declining defense budgets but supporting SOF 51 with conventional forces could reduce the cost of counter hybrid warfare activities if resources are properly allocated (Andreassen, Boesgaard, & Svendsen, 2016; Robinson, 2013). Richardson (2016) concluded that NATO’s counter-hybrid threat mechanism should prioritize hybrid threat education and training at NSHQ and develop a legal framework focused on attribution through a “whole-of-alliance comprehensive approach” (p. 101). The Special Operations Component Command (SOCC). NATO’s post-2014 changes included a rapid action plan to improve the readiness of the NATO Response Force (NRF) (Moon, 2016). The NRF consists of a yearly rotation of a Joint Force Headquarters and component commands responsible for land, air, sea, and NSOF. The NSOF element of the NRF is called the Special Operations Component Command (SOCC), which is a C2 element capable of managing multiple special operations task groups (Dorschner & White, 2014). The NSHQ designed and developed the initial framework SOCC capability as back-up if NSOF was needed and the SOCC was already committed to NATO operation (Dorschner & White, 2014). The SOCC is different from the NATO land command (LANDCOM), air command (AIRCOM), and maritime command (MARCOM) component commands in that it is not a part of the NATO Command Structure and does not have a standing, operational apparatus supporting it (NATO, 2018a). This means that rotational responsibility for the LANDCOM, AIRCOM, and MARCOM segments of the NRF are shared by NATO nations, but there is also a standing command for each, constantly engaged in NATO’s military functions; this is not the case with the SOCC, which is only assembled when the NRF is activated (NATO, 2018a). While the NSHQ developed the SOCC-core as the standard for a NATO SOCC capability, the NSHQ does not exercise C2 over NSOF in rotational NRF duties (NSHQ, 2019c). 52 The NATO Standardization Agreement 2523 established Allied Joint Publication 3.5 as the requirements for NSOF operations (NATO, 2013b). Vores (2012) highlighted the differences between SOF and conventional intelligence with SOF requiring greater detail, precision, and comprehensive fusion of all-source intelligence tailored for each mission. The extensive physical, personnel, and intelligence resources necessary to establish and maintain a SOCC motivated some countries to establish partnerships to consolidate resources into multi-national SOCCs. In June 2017 Denmark, Belgium, and the Netherlands agreed to establish a tri-national SOCC and accept rotating responsibilities as an NRF SOCC (NATO, 2018c). In February 2019, Hungary, Croatia, Slovakia, Slovenia, and Austria announced their intention to combine resources in another regional SOCC (R-SOCC) (NATO, 2019b). James (2016) described the use of the SOCC as the initial C2 element for NSOF in crisis response operations focused on countering hybrid warfare, with intelligence support from the NATO Intelligence Fusion Cell (NIFC) and its special operations component the SOIB. While NSOF would achieve tactical successes, sustained direction, support, and political resolve is required for NSOF to achieve strategic goals (Moon, 2018). Kramer et al., (2018) followed the Brussels Summit suggesting the best asset to counter hybrid threat are small, permanent, fully supported CHSTs of SOF, civilians, LE, and subject matter experts from multiple fields. The intelligence fusion element of the SOCC is the All-Source Center (ASC), which is responsible for collating information from a variety of intelligence disciplines and producing products that meet the detail and depth required to support NSOF missions (Krott et al., 2012). Intelligence fusion in the SOCC ASC aims to optimize NSOF’s concerted intelligence apparatus by drawing from the complementary aspects of all intelligence disciplines (Krott et al., 2012). While one study noted the benefits of combined NSOF training and deployments, it also 53 highlighted NSOF’s lack of organic collection assets and the need and limitations to reach-back to national intelligence agencies for sensitive data (Vores, 2012). This limitation was also noted by Korkisch in 2010 who described national intelligence as a bridge over intelligence gaps supporting tactical operations in Afghanistan. While research by Krott et al., (2012) occurred during the development of the SOCC concept and Russian hybrid warfare was not popularized, it did not clarify how the ASC would manage cyber, exploitation, or information from activities that would today be referred to as gray zone or hybrid activities. Vores’ (2012) research in the same year promoted the use of exploitation and exploitation training at NSOS but did not illustrate how NSOF would capitalize on information from gray zone or hybrid sources. Optimizing Technical Exploitation for the Gray Zone Herion (2012) anticipated the loss of permissive environments in future conflicts and stated the United States will transfer effort to Phase 0, or environmental shaping and deterrence operations and conduct exploitation with Alliance partners. Years later, the United States still struggles to define responsibilities and assign tasks for counter-gray zone activities. Priority research topics and questions for the academic year 2019 at the United States Joint Special Operations University (JSOU) include: a) what are emerging gray zone threats, b) how can the United States Government be proactive in the gray zone, c) what are the mission essential tasks for SOF in the cyber domain, d) how can SOF prepare for cyber-enabled operations, e) how can materials and information gathered by a strike force be used to elucidate relationships between state and non-state actors, and f) identity narratives shaping relations in the gray zone (JSOU, 2019). The success of Task Force 714 in Iraq was well documented and accredited to the shift toward information dominance by thoroughly exploiting information collected from SOF 54 missions (Shultz, 2016). Herion (2012) recognized the value of expeditionary collection and analysis and described the rapid turn-around of collected materials to useable intelligence due to the collaboration between ground collectors like SOF and Weapons Intelligence Teams (WIT) and the presence of Level II or theater-based laboratories. Herion (2012) reviewed the benefit of technical exploitation and expeditious forensic and digital analysis in deployed theater-level laboratories and noted that even with the obvious operational advantages, educating the intelligence community and senior leadership of the benefits of exploitation remains a challenge. By 2010, the DoD had eight expeditionary forensic laboratories supporting NATO operations in Afghanistan, and soldiers conducting TEO in conjunction with these labs greatly enhanced the quality and quantity of raw information from the operational environment (Herion, 2012; Vores, 2012). Conducting triage of digital material in time-compressed situations during SOF missions can aid positive identification, produce follow-on targets, and communicate strategic intelligence (Braccini et al., 2016; Pearson & Watson, 2010; Perry, 2009). However, on-site triage is similarly susceptible to current challenges in the field of forensic science like over-emphasis (San Pietro, Kammrath, & De Forest, 2018). San Pietro et al., (2018) found that growing specializations lead analysts to lose sight of the continuum of physical evidence and recommended a generalist approach toward forensics education and on-the-job training. The United States Army Special Operations Command’s (USASOC) (2019a) trains time-compressed site exploitation in the Operator Advanced Course (OAC), which includes collection and preservation techniques, biometrics, chemical sampling, and digital forensics. The NSHQ brought many elements from the OAC to the NSOS in the TEO course and has trained hundreds of operators and intelligence professionals on TEO procedures (NSHQ, 2019a). 55 The program manager for sensitive site exploitation at the United States Special Operations Command (USSOCOM) stated that SOF need rapid advancements in media exploitation capabilities due to the large quantity of data retrieved from the operational environment and the little time SOF has to exploit it (as cited in Harper, 2017). Materials that cannot be triaged on site may require the advanced techniques and capabilities of a Level II theater-level forensic laboratory (Herion, 2012). The USASOC (2019b) Exploitation Analysis Center (EAC) course is to prepare military and civilian support personnel to perform theaterlevel exploitation in a deployed laboratory environment to include, but not limited to the following functions: chain of custody, digital photography, fingerprinting, trace and bulk chemical analysis, biometrics, and digital media exploitation. The EAC is a deployable, theaterlevel forensic laboratory capable of a wide array of forensic functions from DNA analysis, latent fingerprinting, and digital media extraction and analysis to support SOF (Blinde, 2016). Mobile device & computer exploitation. In November 2017, Russian cyber-criminal Roman Valeryevich Seleznev received a 14-year prison sentence for his participation in an over $50 million-dollar identity theft and bank fraud conspiracy using a digital currency within the now inactive Liberty Reserve (US Department of Justice [DoJ], 2017). When detained in The Maldives, Seleznev’s laptop contained millions of stolen credit card numbers and a list of passwords and online usernames connecting him to years of cyber theft (Krebs on Security, 2017). The value of information retrieved from digital devices is well documented and becomes increasingly important as technology can be used to mask identity (Crist, 2017; Czuperski et al., 2015). Seleznev’s customers utilized encrypted forums, messaging services, and virtual private networks (VPN) to communicate and gain membership into the organization, after which they 56 would establish profiles and access Seleznev’s automated system to buy and sell stolen credit cards through a Liberty Reserve’s digital currency system (DoJ, 2017). According to the director of the United States National Media Exploitation Center (NMEC) Kolleen Yacoub, digital artifacts are the “ground truth” (2018, as cited in Michaels, 2018). The NMEC is managed by the United States Defense Intelligence Agency (DIA) and is responsible for examining documents and electronic files from smartphones, computers, GPS devices, and other digital devices (Michaels, 2018). Typical data found on a mobile phone or laptop can include videos, photos, indications of leadership structure, personal records, family members, birthdates, telephone numbers, email addresses, web browser history, and location data (Michaels, 2018). Most users of mobile devices are not fully aware of the data the devices are capturing. For example, installing new applications on the iPhone will ask the user whether to enable location services. However, iOS 8 and later automatically tracks significant locations and remains enabled in an inconspicuous location until disabled by the user at: Settings, Privacy, Location Services, System Services, Significant Locations (Sorrel, 2018). The NMEC’s success rate of recovering deleted data or data from damaged devices is 60% to 80% and was valuable in analyzing and targeting ISIS networks (Michaels, 2018). At the announcement of NMEC’s acquisition, the DIA solicited requirements to create an authoritative database for the storage and retrieval of DOMEX information throughout the intelligence community, but the status of this database is currently unknown. The convergence of mobile device operating systems to iOS and Android reduced the complexity of mobile device forensics as kernels and file systems are well documented, and their navigation to find digital artifacts is easier (Meffert, Clark, Baggili, & Breitinger, 2017). However, the ease of examining fewer operating systems does not include the increased complexity of examining encrypted or 57 obfuscated data, greater quantities of data, and storage across multiple network layers. Mobile data traffic is expected to grow by 700% between 2017 and 2022 and an estimated 79% of mobile traffic will be video (Cisco, 2019). The number of mobile device apps used to remotely control home appliances, alarms, and personal drones will continue to increase (Llewellyn’s, 2017). The following file extensions accounted for 70.5% of all malicious email attachments: a) .doc, .dot (37%), b) .exe (19.5%), and c) .rtf (14%) (Symantec, 2019). Europol (2018) reported that mobile device malware continues to be under-reported and under-researched but will grow to become a future threat, affecting commercial and government entities. There were at least 63 publicly available exploits for Android devices in 2018, with the trends in Android exploitation indicating, a) growth in operating system and devise specific malware, b) increase in exploits for third party customization settings, and c) initial compromise through memory corruption (Meng, Thing, Cheng, Dai, & Zhang, 2018). Although NATO moves forward with a focus on advanced technology, all of the teams accepting the Coding Challenge at the 2018 TIDE Hackathon integrated Android based solutions into their final deliverables. Unfortunately, there are over 60 publicly available exploits for the Android operating system and research indicates exploit specificity and quantity will increase in concert with Android’s growing popularity and its thirdparty applications (Meng et al., 2018). Network exploitation. Harvey (2017) categorized cyber attack attribution into six categories. The first category is the data source, or evidence of C2 including the transmission of data from a compromised node outside a network and any associated IP addresses, email addresses, malware hashes, and domain names. The second category is malware analysis, or written language, programming language, libraries, malware execution patterns and program 58 stack order, and misspelled words. The third, individual and group attack vectors, or the steps in the full-cycle of compromise from reconnaissance to data exfiltration and system foothold maintenance (Harvey, 2017). The fourth category is motive; what is the purpose of the attack, financial, political, the period of time a system was compromised, and how exfiltrated data was used. The fifth category is correlation with industry, or indications that attacks occur in cycles or patterns that resemble those of a particular industry. Finally, the sixth category of cyber attack attribution, the state of geopolitics and stakeholder priorities in light of recession, economic sanctions, and other financial or national security challenges (Harvey, 2017). Symantec (2019) noted an increase in living off the land attacks, which avoids the use of malware and utilizes available software such as PowerShell. Symantec (2019) noted a 1000% increase of malicious PowerShell scripts in 2018, mostly using Microsoft Office macros to activate PowerShell scripts (Symantec, 2019). Downloading, installing, and running non-native Windows software leaves traces within the Windows registry, however using native Windows tools such as Service Control Manager (SCM) does not (Microsoft, 2018). With access to any computer on a network, the SCM can be used to direct the machine to launch command.exe, which in turn calls PowerShell.exe to execute the malicious script (DeGracia, 2018). This process circumvents most anti-virus software because it does not make registry entries, but instead logs in the system event log with identification number (ID) 7045, which shows newly installed services (DeGrazia, 2018). Opening the system log event ID 7045 shows the malicious binary’s service name, followed by the service file name indicating the use of command.exe (%COMSPEC%) to call PowerShell (powershell.exe) followed by the PowerShell script in encoded in Base64 (DeGrazia, 2018). 59 The increase of data strains physical and cloud-based systems and necessitates intermediate architecture to connect to IoT devices and calculate and store its information (Linthicum, 2019). The extension of processing and storage units closer to the data source is called edge computing, with the principle goal to reduce the latency between the human or device producing data and the central processor, usually a server (Linthicum, 2019). Fog computing outlines the standards for implementing edge computing and consists of switches and routers between the cloud and the device layer (Sandhu, Sohal, & Sood, 2017). The internet of things (IoT). The IoT is projected to include 200 billion devices by 2020 (Chatfield & Reddick, 2019). The United States Governmental Accountability Office (GAO) (2017) defined IoT as Internet Protocol devices interacting with the physical environment containing elements to sense, communicate, and process information. IoT objects can include passive sensors, vehicles, industrial control systems, and other wireless devices that bridge the human to technology gap. The sheer mass of IoT devices and their growing connectivity create vulnerabilities that were previously unknown without extensive research on the deep web (Patton et al., 2014). One Deloitte study (2017) indicated nine of the top 10 economies most vulnerable to IoT attacks are in Europe. The order of vulnerability of these countries is Slovakia, Lithuania, Estonia, Latvia, Czech Republic, Norway, Sweden, Poland, and the Netherlands (Deloitte, 2017). A 2014 report estimated substantial growth in the Industrial Internet of Things (IIoT) market to increase the global economy by over 14 trillion dollars by 2023 (Daugherty, Banerjee, Nejm, & Alter, 2014). Urquhart and McAuley (2018) list healthcare services, food supply chains, mining, and transport and logistics as some of the industries that will heavily rely on IIoT devices within the next five years. Once IIoT vulnerabilities and their associated exploits are identified, they should be classified in a top to bottom hierarchy whereby variations that branch 60 from each level can be isolated and analyzed with finer detail (Boyes, Hallaq, Cunningham, & Watson, 2016). Chatfield and Reddick (2019) identified a lack of extant research on macro-level integration of IoT technology and developed a framework for the United States federal government applying IoT to defense, energy, transportation, and smart cities. Research to enhance automated information processing for the battlefield is no longer science fiction but has expanded from logically acquired information to combine sensory and emotional input through existing IoT technologies to customize soldier’s situational display (Lin, Xia, Li, Wang, & Humar, 2019). Research by Lin et al. (2019) includes logical sensors like a GPS, sound and vibration sensors, and a camera with thermal imaging all on a heads-up display. In addition, the emotion-aware system for the battlefield environment (ESBE) combines physiological and mental data to assess the environment’s collective impact on a soldier and transmit it to commanders for more accurate situational awareness (Lin et al., 2019). Symantec (2019) noted a 78% increase of supply chain attacks in 2018, suggesting complex systems of integrated electronics and code from multiple vendors will increase the overall vulnerabilities of the systems and its sub-components. Drone exploitation. Countering the proliferation of unmanned aerial vehicles (UAV) is a strategic security challenge for Europe that can only be overcome with improvements to international law and the alignment of commercial, political, and military assets (SERA, 2016). The introduction of drones into warfare includes a) the cost and sophistication of image analysis, b) the uncertainty of collating and fusing drone information with other intelligence, c) secure and robust data storage and retrieval systems, and d) the ethical obligation to act on information in violation of international law (Portmess & Romaya, 2015). American Foreign Policy Council Fellow in Russia Studies Samuel Bendett, described Russia’s UAV ambitions as a) initial goals 61 to improve combat precision through coordination with manned artillery or aircraft, and b) longterm goals to possess deep-strike capabilities with larger drones carrying heavier payloads (as cited in Atherton, 2018). Bendett (as cited in Atherton, 2018) described identified four UAVs either currently deployed or in development for the Russian military: the Forepost, a surveillance-only version of the Israeli Searcher, the Orion, similar to Chinese CH-4 and CH-5, and two larger UAVs capable of carrying over 2.5 tons of munitions, the Altius and the Ohotnik. Russian use of UAVs does not replicate the strategy of persistent surveillance practiced by United States and its Allies. Instead, Russian tactical forces and field command units use UAVs to increase the accuracy of artillery (Farley, 2018). Russia has purposefully reduced the size of artillery formations while increasing accuracy using UAVs for terminal guidance. More advanced UAVs like the Orlan-10 and Forpost would identify targets while smaller UAVs like commercial quadcopters flew toward the target for more accurate coordinates (Cranny-Evans et al., 2018). Russian forces in Ukraine exercised command of unmanned aerial vehicles (UAV) both offensively and defensively. Testimony given to the United States Senate Foreign Relations Committee, Subcommittee on Europe and Regional Security Cooperation described Russia’s successful use of electronic warfare to damage the internal circuits of Schiebel Camcopter UAV’s controlled by peacekeeping teams in Ukraine (Blank, 2015). In addition, Russian offensive use of drones to deliver grenades is common. In July 2017, a drone carrying a thermite grenade initiated an explosion at a Ukrainian ammunition depot at Balakliya causing billions in damages (Mizokami, 2017). Similar attacks also occurred at this same location in 2015, and at another depot at Svatovo, which also damaged thousands of nearby homes (Mizokami, 2017). Following an anti-Assad regime attack on a Russian base in January 2018, the Russian military 62 has implemented wide-scale anti-drone training likely using the Kalashnikov-built REX-1 (Izvestiya, 2018; as cited in Tucker, 2018). Devices like the REX-1 are lightweight, man-portable jammers capable of hijacking UAV control signals and blocking automatic recovery by interfering with GNSS ability to send the UAV to its launch destination (Galer, 2018). In December 2018, Russian Defense Minister Sergei Shoigu announced the impending delivery of reconnaissance and attack drones in 2019, including the Forpost, Orion, Altair, Okhotnik, and the Karnivora (Shoigu, 2018; as cited in TASS, 2018). While Russia reportedly experienced a 700% growth of its UAV fleet and aims toward larger, more complex UAVs, there are multiple factors impeding transition from smaller, short-range UAVs to larger, longer-range versions including robust a logistic and maintenance infrastructure, training, and advanced technology that is not readily available (Bendett, 2018; as cited in Atherton, 2018). The size and payload of a UAV will dictate the distance it can be from its target and while Russia’s UAV program is immature compared to the United States, drone strikes with far stand-off would be difficult to attribute to a perpetrator (Floyd, 2018). Native software to personal UAV devices can provide a plethora of digital artifacts from GPS timestamps and waypoints, number of satellites connected, barometer, roll, pitch, distance, azimuth, battery status, video, and photos (Llewellyn, 2017). Open-source software like Dashware, CsvView, Datacon, Wireshark, Network Miner, and Xplico can be used to exploit personal UAVs as demonstrated by Llewellyn’s (2017) examination of a DJI Phantom 3. The use of UAVs for atypical techniques was demonstrated by the United States Army’s use of UAVs running BackTrack Linux to spoof wireless communications, and collect and crack passwords (Thiobane, 2015). The NSHQ is at the forefront of drone exploitation with a 5-day Drone Exploitation course, but while it is a positive step toward advancing digital forensics in 63 NATO, it is unknown how NATO and NSHQ associate this capability to countering hybrid threats or cyber warfare (NSHQ, 2019a). Vehicle entertainment & navigation exploitation. Today’s vehicles save a variety of digital data to include, but not limited to user specific data like contacts, messages, pictures, music, and vehicle specific data such as navigation and vehicle diagnostic data (Le-Khac, Jacobs, Nijhoff, Bertens, & Choo, 2018). According to the United States Department of Transportation (2018), the United States shipped over 52 billion dollars of freight daily, with exports and imports expected to reach 39% of all freight by 2045. The revenue derived from the logistics industry using heavy vehicles creates a greater incentive for adversary attacks as opposed to consumer vehicles (Burakova, Hass, Millar, & Weimerskirch, 2016). The complexity of heavy truck design dictates simplicity when integrating electric and mechanical parts from multiple manufacturers. Heavy trucks in the United States use the SAE J1939 standard for internal network communication (Burakova, et al., 2016). Small, 8-bit microprocessors direct most major functions of an automobile. These electronic control units (ECU) can control a vehicle’s power train, breaks, and general electric components by retrieving information from RAM and delivering it through input/output devices (Ciulla, 2016). Control units vary depending on the functions they support, e.g. fuel injection timers may measure actuator settings thousands of times per second, while other sensors can measure the time doors were open and shut and gear shifting (Selimovic, 2017). The deep web contains a labyrinth of resources discussing cyber attacks on vehicles and adversaries are likely to frequent these sites, especially those discussing high-use or military-grade ECUs (Payne III, 2017). In February 2018, NATO decided to enhance logistics by creating the Joint Support and Enabling Command (JSEC) (NATO, 2018b). In March 2018, Reuters reported the purpose of the 64 JSEC was to increase troop mobility throughout Europe in response to Russian annexation of Crimea. In June of the same year the Russian media company Sputnik reported the creation of the JSEC was in response to the “reunification of the Crimean Peninsula with Russia in 2014” (Sputnik, 2018, para 5). According to Russia, no factual evidence proves Russian involvement in the Ukrainian crisis (Sputnik, 2018). Neither NATO doctrine on logistics nor legacy handbooks address emerging challenges of the logistics industry nor the vulnerability of multipurpose ECU’s throughout NATO’s logistics fleet (NATO, 2012; NSO, 2018e). Civilian use of automobile forensics aids accident reconstruction, criminal prosecution, and facilitates analysis of remote access, automated features, manual interface, and cloud storage (Selimovic, 2017). Research on vehicle forensics conducted by Russian engineers suggested several improvements to vehicle forensic examinations: clarify the existing tools and software used to conduct vehicle forensics and advocate for sufficient diagnostic tools for vehicles with ECUs (Dobromirov, Dotsenko, Verstov, & Volkov, 2017). The extension of fog computing into vehicle-to-cloud communications will reduce the resource requirements and latency of vehiclebased data systems reporting, storage, and retrieval of information in the cloud (Chen, Lu, Xiong, & Xu, 2018). Chen et al. demonstrated the vulnerability of terrestrial mobile sensors and connection protocols to control centers and developed encryption that simultaneously increases security and anonymity while reducing computing and communication overhead. Global financial markets and banks are early adopters of convergent technologies and as the corporate world integrates Artificial Intelligence, blockchain, IoT, and fog protocols to existing computing infrastructure, it is likely the rush toward a payout increases the threshold for accepting cyber risks (Radanliev et al., 2018). 65 Multi-level information sharing. In 2009 the Acting Deputy Chief of Staff for Intelligence for the International Security Assistance Force in Afghanistan, Major General Michael Flynn, attributed intelligence issues in Afghanistan to a lack of ‘grassroots-level’ knowledge acquired from analysts’ interactions with information collectors, not a lack of data or the unwillingness to share. These early mistakes were corrected through the Joint Special Operations Command’s (JSOC) success with fusion centers that combined SOF, intelligence analysts, forensic experts, UAV pilots, and political strategists, to fight the Iraqi counterinsurgency (Niva, 2013). Fusion centers had the challenging task of analyzing networks consisting of multiple sub-networks with overt and clandestine connections, which dynamically modified their structure as the coalition interacted with the operational environment (Shultz, 2016). Flynn (2009) identified the absence of a shared database as a technological challenge to intelligence analysis, but one that could be overcome. A 2014 study on the storage, search, and retrieval of NATO operational records conducted by the NATO Science and Technology Organization found the principle challenges were not financial or technical but were legal and could be overcome through MOUs. Lower tactical echelons and NSOF units do not typically experience the personal division created by over classification, but this trust does not resonate beyond the operational levels of information sharing (Korkisch, 2010; Long, 2014). The need to share intelligence within NATO is a recognized limitation, but recent research of United States federal government’s information-sharing apparatus indicated the technical and classification challenges could be mitigated through personal relationships and collaborative working environments close to the source of the threat (Gardner, 2017). NSOF intelligence sharing and 66 trust begins with shared training at the NSOS and is strengthened through NATO exercises and deployments (Ara et al., 2011). The absence of a shared information system and authoritative database are critical vulnerabilities to intelligence analysis that are exacerbated by data quantity (Weber, 2018). The former Director of Untied States Defense Intelligence claimed it impossible for the DoD to manually analyze terabytes of data daily, while businesses not only process, but profit from big data analysis (Shanahan, 2017; as cited in Weber, 2018). Success during data-heavy deployments requires military commanders and staff to account for multiple, dynamic data streams to include collection feeds, adversary disposition, cyber threat intelligence, supply chain data, and weather data in a common operational picture (Houser & Johnson, 2017). Transition to a data environment where both data quantity and intelligence value are high requires a unified data strategy, an authoritative data source that aggregates data along a recognized ontology, and analysts to synthesize this data into consumable products (Weber, 2018). Future war will more closely resemble an amalgamation of capabilities and activities existing in the gray area between overt conflict and peace, with the principle goal of risk management by (Niva, 2013). Research from the Council on Foreign Relations suggested future SOF operations must consist of flexible, scalable, and tailored groups of SOF and conventional forces with a regional focus (Robinson, 2013). In 2015, the Associate Director for Research at the National Security Studies Institute, Dr. Damien Van Puyvelde, suggested Western leaders focus on navigating the interrelation of threats instead of accurately defining and potentially oversimplifying hybrid warfare. Experts at Black Hat Europe 2018 suggested intelligence led security be defined as the collation, evaluation, and integration of data to understand cyber 67 threats, mitigate damage, attribute action to actors, and predict future activity (Van der Walt & Pillarisetty, 2018). Following the 2016 Warsaw Summit NATO Headquarters offered MOUs to Allied nations to facilitate information sharing of cyber-related intelligence without a fully ratified policy; as of 2017 twenty-two nations had established MOUs (Shea, 2017). NATO renewed relationships with STRATCOM, the NATO CCDCOE, and the European Centre of Excellence on Countering Hybrid Warfare, and signed bilateral agreements with Georgia, Moldova, and Ukraine to share hybrid warfare and cyber-related information (Maronkova, 2018). Improvements in intelligence sharing is not an uncommon topic in NATO Summit declarations but imbedded in the 2018 Brussels Summit declaration is the assumption that improved intelligence sharing is a holistic endeavor with unspecific requirements for collection, transmission, storage and retrieval (Moon, 2018). Following the Brussels Summit, Senior Fellow of the Center for Asymmetric Threat Studies at the Hybrid COE Dr. Gregory Treverton (2018) argued for enterprise-level, society-comprehensive sharing of hybrid intelligence. In 2012, Vores identified the requirements for NATO SOF intelligence officers deploying to Afghanistan to support ISAF SOF in the SOFFC. Vores (2012) suggested the requirements for United States SOF and NSOF intelligence officers were similar and recommended NSOF intelligence officers attend a series of NSOS courses including SOCC staff and planning courses, intelligence courses, and exploitation courses. Vores’ 2012 study on NSOF intelligence training suggested most students attending training at the NSOS were marked for leadership positions. Yun, Faraj, and Sims (2005) noted an authoritarian leadership style produces efficient results in high-reliability organizations (HRO) during crisis, while participative leadership may have the same effects, but with a time penalty. Research by Yun et al. (2005) focused on high functioning, 68 but temporary teams like emergency room workers, but did not account for HROs with highlyskilled, mature teams with advanced levels of collective mindfulness like military organizations. Pearson (2018) addressed the gap in Yun et al. (2005) and found that HROs like military and LE function best when leaders direct task integration but participate to enable followers when engaged in critical tasks. Technical exploitation and attribution. Identity is a fluid concept affected by environmental stimuli over time, with both contextual and circumstantial peculiarities. Identity attributes like biology, psychology, and society impose conscious and unconscious constraints on an actor and can be categorized and fused with other information to predict behavior (DoD, 2016b). International forces operating in complex environments can benefit from sharing carefully controlled, unclassified identify information (DoD, 2016b). The NSHQ innovated exploitation within NATO by developing an organic exploitation program, facility, and training courses to enable capability and capacity growth within NSOF (Ara et al., 2011). The NSOF’s exploitation courses focus on tactical and operational levels and were specifically designed to augment the lack of national-level intelligence provided by NATO nations by acquiring and training commercial off the shelf equipment (Ara et al., 2011). All networks have critical functions and vulnerabilities that provide an adversary the opportunity to exploit (Cullen & Reichborn-Kjennerud, 2017). Synchronized hybrid activities focused on a network’s vulnerabilities produce non-linear results whereby their analysis cannot draw direct causality and their effects cannot be accurately predicted (Cullen & ReichbornKjennerud, 2017). To that extent, digital forensic analysis will not present irrefutable attribution between an individual and activity, but the totality of circumstances will increase the association 69 probability between and attacker and a digital artifact, i.e. reducing the length of the technology to biology bridge and avoid assumptive associations (Pearson, 2013). Quality attribution is imperative to international policies of deterrence (Rid & Buchanan, 2015). Attributing cyber incidents requires analysis of the attacks’ goals along the operational spectrum (Rid & Buchanan, 2015). The analytic goal at the tactical level is to understand the technical aspects of infiltration and compromise. The analytic goal at the operational level is to reconstruct the attack’s architecture and the attacker’s profile, to include similarities to existing malware payloads. The analytic goal at the strategic level is categorize the motivation and purpose behind the attack and its immediate and long-term intent, whether economic, political, or other. (Rid & Buchanan, 2015). Lists of malicious digital signatures and other components of malware payloads are available from the National Software Reference Library (NSRL) and commercial security companies who produce blacklists that can be loaded on intrusion detection systems at the operational level and on forensic machines to create watchlists of malicious artifacts (Mead, 2009). Core forensic processes include authentication, identification, classification, reconstruction, and evaluation (Pollitt et al., 2019). Identifying malicious artifacts and behavioral data in conjunction with specific questions and actions during each of these processes helps minimize assumptive associations (Pollitt et al., 2019). Appendix A includes selected forensic questions from Pollitt et al. (2019). Discussion of the Findings The purpose of this research was to examine the role of technical exploitation in nonArticle 5 crisis response operations, specifically NSOF’s use of tactical digital forensics to provide operational and strategic intelligence to counter gray zone to hybrid warfare activities. What are NATO’s requirements for collecting cybersecurity information? How should NSOF 70 optimize exploitation’s procedures and tools to identify forensics artifacts relevant across the operational spectrum? What are the benefits of a framework for cross-operational sharing of cybersecurity and digital forensics information? Anticipating Gray Zone Activities The shift to environmental shaping and deterrence operations has become a reality and NATO must delineate responsibilities and properly resource counter-gray zone activities. The RAND Corporation concluded that Russian military capacity has not sufficiently improved as to provoke near-peer conflict without transparent threat to its sovereignty (Boston & Massicot, 2017). Russia’s military strategy is power projection to deter NATO from interfering with Russian affairs, not to engage in protracted war (Thornton, 2017; Kyle, 2019). This supports the premise that NATO’s operational planning for crisis response should focus on countering gray zone to hybrid operations as opposed to augmenting conventional military capabilities (Boston & Massicot, 2017; Mattis, 2018). Russia will utilize APT groups along the spectrum of conflict and take great effort to obfuscate attribution. While APT groups are capable of developing zero-day attacks, bulk resources will be allocated to modify existing exploits and perform living off the land attacks through Microsoft PowerShell; development of zero-day malware will decrease and be reserved for high-profile targets (Symantec, 2019). These cyber activities will consist of highly sophisticated, targeted reconnaissance and attacks on critical infrastructure with emphasis on the energy sector. Russian use of energy coercion is a well-demonstrated strategy but has been limited to short-term disruptions. While research suggests Russian long-term energy coercion is not economical, this limitation will decrease with the development of alternative gas pipelines in the Baltic Sea and Siberia. 71 Global freight is a multi-billion-dollar industry and Russia will seek to compromise vehicle ECUs to interfere with logistics within the Baltics and other European, non-NATO countries like Moldova, Ukraine, and Georgia (Dobromirov et al., 2017). Russia will likely monitor Western military research and development, and attempt to compromise components of commercial systems through the supply chain (Symantec, 2019). Supply chains attacks increased 78% in 2018 (Symantec, 2019) and compromise of vehicle-based information systems will expand to fog computing sensors when they are implemented by commercial companies to optimize metadata transfer and storage along logistic routes (Chen et al., 2018). NATO exercised logistic support during the 2018 Trident Juncture in Norway, but it is unlikely that NATO logisticians are working closely with NATO’s Cyber Operations Center to counter logisticsrelated cyber vulnerabilities, and while the extent of similarity between vehicle ECUs is outside the scope of this research, European land-based logistics is a logical target for Russian malware. Malware distribution through email attachment will remain a staple component of Russian APT groups’ attack vector. The development and availability of inexpensive Russian software could easily be propagated through non-Russian software companies based in Europe. The distribution of malware through social media and third-party applications on the Android OS will increase and spread to IoT devices (Meng et al., 2018). Nine of the 10 countries most vulnerable to IoT attacks are within Europe (Deloitte, 2017). This information combined with the estimated worth of the industrial IoT market to over 14 trillion dollars by 2023 will contribute to exploitable vulnerabilities within the industries reliant on IoT devices to include the energy and logistics sector (Urquhart & McAuley, 2018). Current research does not indicate organized criminal syndicates are responsible for the preponderance of cyber crime, but Internet-based crime will be embraced by governments, 72 criminals, and any individual or group possessing the will to engage in regular criminal activity. Russia will capitalize on criminal networks within Allied countries and outside of Europe to a) reduce resource consumption, b) reduce attribution, and c) increase financial opportunities. Connectivity between Russian political and military leadership and organized crime groups is likely more prevalent than reported and the presence of ethnic Russian populations outside of Russia are vulnerable to recruitment and extortion. Specifically vulnerable are ethnic Russians working for technology companies, financial institutions, the energy sector, or local governments outside of Russia. The RIS is operating within ethnic-Russian populations globally to promote Russian state goals through the use of gray zone activities best suited for that environment. The advice of Russian information warfare theoretician Igor Panarin was clearly implemented as Russia demonstrated vast improvements of their command of the global information space between the conflict with Georgia in 2008 and Ukraine in 2014. Thornton described Russian military strategy to catalyze disunity both within nations and their alliances. Russia has not hesitated to interfere with several national and parliamentary elections to include promoting intra-national political dissension through the use of APT services and the dissemination of propaganda through complex botnets in countries politically aligned with Russia. This was most evident during the 2016 United States presidential election and the expansive use of computational propaganda from Venezuela to promote Catalan independence from Spain in 2017. Russia will attempt to infiltrate political systems in countries whose location or economic potential can be leveraged, e.g. Venezuela, and Nigeria. Those countries with updated or homogeneous voting systems combined with a lack of government oversight will be most vulnerable to election interference as the procedures for widespread compromise are simplified when system-wide hardware and software components are identical. 73 Russian IO campaigns consist of mass propaganda, disinformation, and access control to non-state sponsored media in Russia and its occupied territories. Russian media will advertise Russian advances in science, technology, and military capability without international verification, which is an exploitable weakness in the Russian narrative. This was evident by Vladimir Putin’s claim to welcome a Cuban Missile-like crisis followed by Russian media announcing two closed United States military bases were potential nuclear targets. While Russian-language IO campaigns softened the impact of Crimean annexation, research shows the ethnic Russian population’s response to IO will vary between countries and heavy IO campaigns in the Baltics will not likely galvanize the population as it did in Crimea (Kyle, 2019). Russianspeaking areas of the Baltics are vulnerable to strategic human intelligence collection and recruitment, and the RIS is likely to engage criminal enterprises within these areas as proxy for surveillance and reconnaissance in what would otherwise be denied territory. Giles described IO as information-psychological and information-technical. The Atlantic Council’s DFRLab is the lead organization tracking disinformation and should be leveraged to provide the bulk of information-psychological information in conjunction with other assets like STRATCOM. Information-psychological data is centered on strategic political messaging and its analysis is outside the scope of intelligence support to NSOF and is more likely to be observed or collected by open-source analysts in the aforementioned organizations. To clarify this point further, the author is not claiming intelligence support to NSOF is disconnected from psychological operations, information warfare, or intelligence frameworks like political, military, economic, information and infrastructure (PMESII), but stating that political policy analysis is not the job of an NSOF intelligence analyst. Information-technical IO should consist of aggregated data from any NATO, partner, and commercial organization capable of collecting 74 tactical cyber intelligence to analyze technical activity and artifacts along the cyber kill chain (Lockheed Martin, 2019). Intelligence collection in hybrid conflicts will be greatly reduced compared to the operating environments of Iraq and Afghanistan. This is due to the adversary's ability to occupy and control territory to deny entrance to coalition aircraft and ISR assets. Russia’s large army could rapidly seize and hold land in neighboring countries if it desired and would immediately be followed by placement of anti-aircraft systems to deny airspace (Bugajski, 2016). Russian freedom of movement in buffer areas like Eastern Ukraine, Crimea, areas of Moldova facilitates the rapid fortification of seized territory. Russian electronic warfare capabilities are highly competent and Russia’s use of smaller UAVs to improve the accuracy of field artillery is effective. Russia strives to develop larger, more capable surveillance UAVs but technical and economic challenges will impede operational fielding for several years (Bendett, 2018; as cited in Atherton, 2018). Russia has demonstrated the effective use of GNSS jammers, and although the Russian military uses GLONASS to for navigation and precision targeting, it is unknown if Russia possesses jamming equipment sophisticated enough to jam the L2 GPS band without interfering with the GLONASS G2 band. Since GPS operates on three frequency bands, an attempt to jam any one of GPS’s bands is unlikely to deny GPS service, but reduce it. However, there is enough bandwidth separation between most GPS, GALILEO, and GLONASS bands to jam GPS and GALIELEO effectively without affecting GLONASS. Both NATO and countries neighboring Russia have expressed concern of Russian interference with commercial and military use of GPS and NATO is engaged in projects to geolocate jamming devices. While this is a positive step, effective DOA geolocation requires either a mobile receiver or a wide array of stationary 75 antennas to produce accurate resolutions, still these would not likely resolve to less than one kilometer, depending on the distance between the emitter and receiver. Revising Doctrine Russian leadership is conscious of the disadvantages of a conventional war with NATO and will not deliberately violate Article 5, but it will utilize gray zone tools to probe the Article 5 threshold to assess NATO’s political will and retaliation. Treverton suggested the emergence of hybrid threats provides the intelligence community an opportunity to step away from the mechanistic implementation of traditional cycles, refocusing efforts on clearly identified requirements including those derived from cyberspace. Literature discussing hybrid warfare at the strategic level is extensive (FireEye, 2016; Johnson, 2013; Korkisch, 2010; McCulloh & Johnson, 2013; SERA, 2018) but few studies described the tactical employment of counterhybrid warfare or counter-gray zone activities. NATO is an international organization whose capabilities are determined by the contributions and priorities of its 29 member nations. Adaptation to the rapidly changing digital world and the role of warfare is limited by national perspectives and national contributions. Estonia, for example, places a high value on cybersecurity compared to other countries whose national concerns may be more directed toward countering Islamic extremism from North Africa or the influx of Syrian refugees. Any one country's capabilities may be more developed than the collective NATO capability and the Alliance’s ability to counter gray zone activities rests largely on a collaborative network of all-source intelligence. Part of the United States DoD plan to address cyber deterrence challenges included and enhancing existing capabilities (DSB Task Force on Cyber Deterrence, 2017). The value of exploitation has been highlighted by NATO and the NSHQ since inception, but the potential to 76 collect valuable digital artifacts of strategic value during TEO is consistently underemphasized. Exploitation programs of varying levels of maturity exist within NATO SOF units and the connection between TEO and cyber intelligence requires clarification and greater attention. The absence of a unified cyber command is a weakness for the Alliance and the United States Cyber Command can be referenced as a framework. Strategic requirements must be translated into focused, tactical priorities, whereby SOF tools are customized, and missions are planned around the need to collect digital artifacts and cyber related information. At the strategic level, intrusion detection systems and defensive mechanisms are in place to protect NATO's communication infrastructure, but these relate specifically to IT security, not the translation of various cyber activities to all-source intelligence products. Analytic capabilities to understand digital forensics and cyber intelligence in the greater scope of warfare are immature throughout the Alliance. Cyber exercises Crossed Swords and Locked Shields are invaluable to test NATO’s cyber resilience. While specifics of the exercises are not published, the presence of digital forensics experts suggests the exercises contain an element of digital data extraction and analysis on drones, mobile phones, computers, servers, or other IoT devices that provides some clarity on an attackers’ location, intent, or attack vector. A data aggregator and visualizer like the ADAM and EVE system used in Locked Shields 2017 are critical components of a comprehensive operational picture of cyber intelligence. While detailed research on this system is not available, the ability to access EVE through HTTP POST suggests that any identifiable string, section of code, or hash that resides in ADAM’s taxonomy as a security threat, could be identified during forensic examination and sent to EVE via the web, contributing to the operational picture. 77 Technical exploitation produces raw information consisting of biometrics, forensics, and DOMEX and NATO should endeavor to conceptualize and develop a unified exploitation system that produces a series of intelligence products aimed to reduce threat anonymity. The preference of biometrics over other forms of exploitation is evident through both NATO standards (NSO, 2013a) and NATO doctrine for biometrics in AIntP-15, and while this same preference exists in the United States, the DoD assigned executive agents for exploitation disciplines and each are properly resourced. NATO is heavily focused on biometrics, which severely limits its ability to identify gray zone, hybrid, and cyber indications and intent within a tactical environment; biometrics will confirm an identity, but digital forensics provides context to adversary intent and capabilities. National perspectives on human identity and privacy vary, and while biometrics can establish direct connections between and individual and a physical object, the utility of biometrics is reduced as the distance between a perpetrator and victim increases. The introduction of the term Forensic and Biometric Intelligence (FABINT), as suggested by the NATO SPCOE, unnecessarily complicates the already underutilized domains of exploitation, which are outlined in Technical Exploitation doctrine (NSO, 2015). Mancini et al., (2017) justified the value of data collected without exact forensic standards and Pearson’s (2018) contribution emphasized the impact of time on exploitation operations suggesting proper evidentiary procedures are only deliberately avoided under time and safety restrictions. There is no need to combine or rename exploitation domains when exploitation will remain driven by commanders’ information requirements, the mission goal, and situation on the ground. To exacerbate confusion of terms and authorities, there is no clear distinction between a single NATO-run exploitation system, but there are inferences that three separate systems exist between stability policing, counter-IED, and TEO (SPCOE, 2016). 78 NSOF leads CHSTs. The preponderance of research indicates special forces are best suited to counter gray zone activities within Alliance countries, in partner nations, and in occupied territory if necessary (Andreassen et al., 2016; Breede, 2018; James, 2016; Moon, 2018; Shamir & Ben-Ari, 2016; Vores, 2012). Special operations teams are likely to be a primary resource to infiltrate enemy territory for intelligence collection, but their ability to exploit adversary communication systems for intelligence value must equal the sophistication of their adversaries' capabilities (Long, 2014). Exploitation is one of NSOF’s greatest assets to reduce ambiguity within the operational environment and increase the probability of attribution, especially during highly sensitive missions. While exploitation is doctrinally defined in AIntP-10 on Technical Exploitation, the amalgamation of research did not show clear connections between tactical exploitation doctrine and its relevance to counter gray zone activities, hybrid threats, or indications of cyber activities. The absence of these connections in research demonstrates a gross misunderstanding of exploitation’s contribution to the aforementioned activities. Technical exploitation will inevitably produce hidden gems of strategic value if analyzed in conjunction with Russia’s use of hybrid warfare in each specific operational environment and the relationship of these tactics to Russia’s National goals; however, these strategic gems will need to be isolated from the larger quantity of tactical intelligence linked to the intelligence cycle (McKew, 2017). Moon (2018) described the benefit of allocating NSOF C2 to regional SOCCs to consolidate and fuse a holistic representation of gray zone and hybrid developments. Unfortunately, the SOCC is not a standing component of the NATO Command Structure and is only assembled when the NRF is activated. Stand-by responsibility for the NRF SOCC changes annually and even when in stand-by, a SOCC is not fully assembled and working toward a shared mission as are the three other NATO tactical commands. Moon’s (2018) suggestion to 79 allow theatre or regional SOCCs to lead counter-hybrid threat activities is valid only if established upon a stable core of national specialists with support from subject matter experts who rotate every few years (DeTrinis, 2017; Kramer et al., 2018). In 2012 Russian hybrid warfare and cyber warfare had not yet ascended to the top of NATO’s priority list, making it difficult to envision how the SOCC ASC would manage cyber, exploitation, or information from activities that would today be referred to as gray zone or hybrid activities (Krott et al., 2012). Training basic digital forensics is more complex than training basic intelligence, and the ASC needs a bi-directional, technical-to-analytic, and analytic-to-technical approach to ensure exploitation information is properly managed. An exploitation manager in the ASC must be an experienced intelligence analyst and a highly trained exploitation professional and be accompanied by a team of technicians and analysts who can process and report technical exploitation information for a general audience. Digital forensics produces a plethora of raw, unclassified data that is not bound by national dissemination caveats like traditional intelligence disciplines. Raw information derived from a nation's SIGINT and HUMINT capabilities is seldom distributed to Alliance members without a strict editing protocol to protect information sources. Enhancing digital forensics capabilities throughout NATO gives the Alliance access to a plentiful, unclassified data source, but the complexity of digital forensics and cybersecurity contribute to their lack of attention. Integrating Exploitation Capabilities Strategic and operational support. For NSOF to capitalize on digital artifacts and counter gray zone activities strategic CCIRs must be translated into datasets that are managed at the operational level and made available to NSOF and CHST members on the ground. These datasets should include, but not limited to artifacts from: networks and their periphery, critical 80 infrastructure mobile phone and IoT devices, critical infrastructure processor control units, vehicle ECUs, GPS jammers, UAVs, APT malware, general computer usage, and social media. The NIFC, NATO JISD, SOIB, NATO Hybrid Analysis Branch, NATO Cyber Operations Centre and other NATO intelligence elements should maintain a gray zone/hybrid intelligence preparation of the operational environment (HIPB) which is dynamically modified as adversary tactics, techniques, and procedures are adjusted. Both information-psychological and information-technical data should be fed into the NATO Cyber Operations Centre, which also has connectivity to comprehensive NATO intelligence. The Hybrid COE and The DFRLab are competent in tracking disinformation and should work in concert with NATO STRATCOM to consolidate information-psychological IO for consumption at the strategic level. The Hybrid COE and NATO Hybrid Analysis Branch should also work closely with the NATO JISD and SOIB to identify the types of gray zone activities used by the adversary while the Cyber Operations Centre and the CCDCOE identify the information-technical data coming from those activities; ensuring this data is separated into a) operational formats, consumable by intrusion detection systems, and b) tactical formats, consumable by TEO equipment used by NSOF or CHST members. This technical data should be consolidated and presented in a digital format whereby a new NATO exploitation cell converts this input into a configured watchlist including hash values, digital signatures, indicators of compromise, etc. and loaded to TEO equipment by NSOF operators. Inevitably, the CHSTs will face circumstances requiring additional exploitation and intelligence support. While CHSTs should include of a small, highly capable intelligence team, there should be connectivity to garrison intelligence from the NIFC and SOIB. Complex exploitation exceeding the capabilities of the CHSTs will require Level II, or theater-based 81 exploitation from a forensic laboratory like the European Defence Agency’s forensic laboratory (JDEAL). The JDEAL is counter-IED focused and this research found limited information describing its operational requirements, maintenance, and deployment. Nonetheless, the JDEAL and other available Level II assets are critical to the exploitation process and the utility of TEO within a CHST. The significance of a Level II exploitation capability cannot be understated, and their value is well documented in the Iraq and Afghanistan conflicts. While a Level II exploitation laboratory can be manned with NSOF, conventional forces, and LE, the depth of sophistication of Level II equipment and techniques dictates a higher level of training and knowledge of the application of exploitation across the operational spectrum. Exploitation personnel within CHSTs will not have the time and experience to manage Level II exploitation activities, thus the operating of a Level II capability should include input from the C-IED COE, JDEAL, and the United States DFBA. TEO within the CHSTs. The intent to use multi-functional counter hybrid support teams is valid if properly resourced with experienced and highly trained individuals from multiple disciplines who are allowed the authorities and autonomy to interface with national and local military and LE elements. European militaries do not traditionally integrate with LE and digital forensics capabilities with NATO member countries' militaries are limited due to a combination of the cost and complexity of training and the price of commercial tools. Additionally, Brown (2016) noted the general lack of technical and human resources of local LE units to handle cyber crime investigations, thus a robust TEO capability within a CHST may serve to facilitate local investigations while simultaneously providing information on gray zone activities to include indicators of compromise, political strategies, and technological capabilities and limitations. 82 Organizations like the MP COE and the SP COE should train and support NATO police experts to accompany NSOF and facilitate exploitation from a European LE perspective. The NSHQ’s TEO program and exploitation training classes provide a baseline to promote NSOF exploitation within CHSTs through equipment loan and training. The NSOS has established Digital Media Exploitation, Cellular Phone Exploitation, and Drone Exploitation courses that predominantly use open-source tools and allow students to replicate forensic procedures learned during the course without cost (NSHQ, 2019a). The NSOF’s exploitation courses focus on tactical to operational-level information to feed the intelligence cycle because strategic requirements for exploitation have not been communicated. This researcher found no existing literature illustrating exploitation’s connectivity to operational and strategic indications of gray zone activities, while the analysis within this Capstone demonstrated the potential value of digital artifacts toward this end. Upon the development of counter-gray zone strategic requirements for TEO, the content of NSOF’s exploitation courses could be analyzed and enhanced to attribute cyber and other gray zone activities and facilitate analysis to understand the adversary’s motivation and capabilities. Additional iterations of these classes could be delivered; to include NSHQ-supported mobile education teams to CHST locations. Exploitation analysis. The biology to technology bridge is built through effective exploitation analysis; however, this form of analysis is lower in priority compared to traditional intelligence forms in NATO. Biometrics, forensics, and DOMEX in addition to behavioral patterns of life contain the majority of attributes that constitute identity, which is the key to attribution. The sensitivity of personal identity and privacy requires precaution, thus leading to a necessary terminology change from the United States I2 to the new NATO exploitation analysis. Exploitation should be recognized as an intelligence discipline and exploitation within NATO 83 should be unified. Exploitation analysts are the first line in connecting tactical gray zone information to operational commanders and exploitation analysis should be integrated into future NATO intelligence products through comprehensive information fusion in the field and in garrison. Investigation on intelligence sharing in fusion centers illustrated the characteristics of an effective counterterrorism framework from the United States Federal to local levels (Gardner, 2017). This work can be extrapolated to either introduce gray zone/hybrid cells within existing intelligence units or augment deployed forces to assist with the collection and dissemination of gray zone information from digital artifacts. Limitations of the Study The concepts of hybrid warfare and cyber warfare are not new and hundreds of academics, technicians, military members, and political scientists have contributed to the existing body of knowledge. This study references, but does not include, detailed information from documents classified as NATO Unclassified and United States for Official Use Only, while most of this information is readily available on the Internet. Details of these references were excluded to respect document dissemination restrictions, protect Allied information, and to produce a product with objective results although more succinct information was available with the aforementioned classifications. In addition, this research did not include interviews with NATO personnel, military, civilian, or retired, thus may incorrectly represent current or planed policy, structure, or operational relationships. The complexity of these topics required extensive introduction to hybrid warfare and gray zone activities, NATO requirements for hybrid and cyber threats, and NATO’s exploitation infrastructure. The lack of research and visibility of NATO exploitation and its capabilities necessitated thorough introductions to the two former topics to bridge the gap between the 84 challenge, identifying indications of gray zone and hybrid activities, and the solution, linking them to strategic significance through exploitation. This researcher is aware that separate sections of this Capstone could constitute volumes of text with more relevant topics and data to support or contradict the product conclusion. However, this reality justified macro-level analysis to aggregate seemingly disparate and highly complex topics to support a comprehensive strategy. Future Research and Recommendations Future research on gray zone activities and their strategic value should be framed like the 2018 SERA panel where several multi-disciplined committees of subject matter experts reviewed one sub-topic of a larger conference theme. This framework would allow greater depth into a) the development of strategic policy, b) operational guidance and management of operationallevel counter gray zone, hybrid, and cyber activities, c) tactical collection and exploitation, and d) the infrastructure to store, retrieve, analyze, and disseminate threat intelligence for strategic effects. Information sharing is imperative to countering 21st-century threats, but NATO should consider operational security prior to publishing detailed research on counter-strategies and tactics at the unclassified level. NATO and non-NATO partners should align with academia and industry to produce classified research specifying a holistic apparatus whereby tactical-level operations align with CCIRs and the exploitation output from tactical missions satisfy commanders’ requirements along the operational spectrum. This new research should include comprehensive analysis of the authorities and responsibilities for doctrine and training of Level I exploitation on-target, Level II exploitation or theater-level forensic laboratory support, and the analytic techniques to translate raw indicators of gray zone, hybrid, and cyber activities into consumable intelligence products. While exploitation can occur during conventional policing, high-risk NSOF missions, and other NATO-led operations, the assignment of exploitation 85 authorities should not remove or reduce the capabilities of operational and tactical units but relieve the economic burden of exploitation training and clarify the mechanisms by which exploitation information contributes to the development of counter-gray zone strategies and their execution. New Research Question 1: How should technical exploitation be integrated into existing intelligence products? Pollitt et al. (2019) suggested that incorporating digital forensics into a unified concept of forensic science would facilitate the association of an individual to a virtual profile and clarify the relevance of digital artifacts to decision-makers (Pollitt et al., 2019). Virtual profiles, digital behavioral patterns, and digital artifacts are already used to attribute malware to APT groups and while digital forensics may not immediately provide confirmatory evidence like biometrics or DNA, the availability of this data and the totality of circumstance can bridge the biology to technology gap. New Research Question 2: What analytic procedures can facilitate the attribution of gray zone activities to individual(s), groups, or nations? Hybrid warfare practitioners will utilize the existing resources of criminal organizations to infiltrate denied territory and accomplish operational goals (Schroefl & Kaufman, 2014). Social relationships and the environment impart their influence on human behavior and shape most criminal theories (Sierra-Arevalo & Papachristos, 2015). There is a direct connection between criminal activity and the communities in which they operate, and social network analysis (SNA) can be used to understand these relationships (Kadushin, 2012). Human networks can be analyzed using SNA, which is an expansion of graph theory and matrix algebra to represent the interdependence of nodes within a network (Borgatti, Everett, & 86 Johnson, 2013). Research showed that crime and victimology are not random, but there are a myriad of social connections of varying type and strength that contribute to individual roles in criminal events, e.g. offender, co-offender, etc. (The International Association of Crime Analysts (2018). Twenty-first century policing will depend on data analytics and methods like SNA to help structure LE strategies (McHale, 2015). Social network analysis as applied to criminal analysis may enhance intelligence analysis of gray zone activities. Additional research on analytic techniques to attribute gray zone activities should include review of: a) CARVER 2.0 (Greaver, Raabe, Fox, & Burks, 2018) and b) Reverse Intelligence Preparation of the Battlefield (Feltey & Rae, 2018). In Spring 2016 the USSOCOM awarded Palantir Technologies a 222-million-dollar contract for its all-source fusion software platform, Palantir (DoD, 2016a). This figure was nearly quadrupled on March 27, 2019 when the Pentagon awarded Palantir a contract for 800-million dollars and was granted the title of program of record (Fazzini & Macias, 2019). It is unknown if NATO will also adopt Palantir as its intelligence platform, but what is certain is that interoperability between the United States intelligence community and NATO intelligence is mutually beneficial. New Research Question 3: How can gray zone activities be categorized across a spectrum of conflict to guide resource allocation? Complex cyber and technical information is not easily translated to the immediate impact on a leader’s goals. Models like the Lockheed Martin Cyber Kill Chain (2019) and hybrid tools (Treverton, 2018) should be combined to create matrices outlining the digital artifacts found while examining indicators of compromise like those indicated in NCCIC cyber activity alerts (2018). These digital artifacts are not exclusive to network compromises but can also include signatures of known vulnerabilities in software and hardware, such as the Ryzenfall 87 vulnerability in AMD microprocessors (Cimpanu, 2018a). This data must be packaged by strategic or operational assets in conjunction with the format required by tactical exploitation equipment. Once this blacklist of digital signatures is compiled, NSOF could quickly configure exploitation equipment and triage digital media for rapid dissemination of threat indicators. The author suggests additional analysis and improved infographics consolidating gray zone activities along a spectrum of conflict. Figure 5 is an adaptation of Hoffman (2016) and Treverton (2018) and is not intended to illustrate accuracy, but the usefulness and direct assimilation of information through quality infographics. Figure 5. Gray zone and hybrid tools with distribution. Recommendations As Fábián (2016) suggested, SOF’s unique traits make them the ideal vehicle to lead NATO’s adaptation to offset Russian hybrid warfare with customized technology as opposed to a reflexive, technically conventional doppelganger. Specifically, NSOF’s ability to reduce ambiguity in the operational environment through the use of advanced TEO and digital forensics is underemphasized, under-resourced, and absent from strategic discussions of countering gray zone activities. Heavy emphasis on biometrics and the quantity of organizations staking claim to exploitation detract from exploitation’s overall utility as it is only partially represented by each and safeguarded to retain the identity of the organizations designed to promote it. 88 Counter hybrid support teams must be educated and capable of performing basic triage of digital artifacts from a variety of sources followed by the submission of structured exploitation data to an authoritative exploitation database. This database should be one component of a larger intelligence system that includes tactical-level cyber intelligence feeds. The quantity of data within digital devices necessitates a triage that is dependent on strategic and operational requirements. Tactical exploitation without triage requirements will produce large datasets that are illegible to all-source analysts. Strategic and operational cyber requirements must be identified, categorized, and packaged into consumable datasets that NSOF can use to configure exploitation equipment. Exploitation is the dark matter of the intelligence world; keeping things together while little attention is paid to its existence. Individual and disassociated pieces of intelligence can hold great significance and may carry an operation, but exploitation will provide the vertices by which gray zone activities are connected to strategic intent. Exploring this path will prove the most economic and efficient way to a comprehensive counter-gray zone strategy for NATO. 89 References Aftergood, S. (2014, July 30). Identity intelligence and special operations. Federation of American Scientists. Retrieved from https://fas.org/blogs/secrecy/2014/07/identity-intel/ Alandete, D. (2017, November 11). Russian Network Used Venezuelan Accounts to Deepen Catalan Crisis. El Pais. Retrieved from https://elpais.com/elpais/2017/11/11/inenglish/1510395422_468026.html. Allied Command Transformation. (2018) TIDE hackathon final report. NATO. Retrieved from http://www.act.nato.int/images/stories/events/2018/tide-hackathon/2018-tide-hackathonreport.pdf Allied Command Transformation. (2019, September). TIDE Sprint. NATO. Retrieved from https://www.act.nato.int/tide-sprint Andreassen, J., Boesgaard, K., & Svendsen, A. (2016). NATO needs to better integrate conventional and special operations forces. In Skinner, E. (Eds.) Countering hybrid warfare: The best uses of SOF in a pre-Article V scenario. CTX, 6(4), 83-89. Retrieved from https://www.ffi.no/no/Publikasjoner/Documents/CTX_Countering Hybrid Warfare. The best use of SOF in a pre-article V Scenario.pdf Applebaum, A. (2018, January 18a). Russia Finds Young Men Who Love Guns and Grooms Them. Washington Post. Retrieved from https://www.washingtonpost.com/opinions/global-opinions/russia-finds-young-men-wholove-guns--and-grooms-them/2018/01/18/0090fbb2-fbd1-11e7-a46ba3614530bd87_story.html?noredirect=on&utm_term=.a098c90a1396 90 Applebaum, A. (2018b). The black hole at the heart of NATO. The Wilson Quarterly, Fall 2018. Retrieved from https://www.wilsonquarterly.com/quarterly/the-fate-of-the-internationalorder/the-black-hole-at-the-heart-of-nato/ Ara, M. J., Brand, T., & Larssen, B. A. (2011). Help a brother out: A case study in multinational intelligence sharing, NATO SOF (Master’s thesis). U.S. Naval Postgraduate School, Monterey, California. Retrieved from https://apps.dtic.mil/dtic/tr/fulltext/u2/a556078.pdf Arquilla, J., & Ronfeldt, D. (1996). The Advent of Netwar. RAND Corporation. Retrieved from https://www.rand.org/pubs/monograph_reports/MR789.html Ashford, W. (2018, August 2). Three Carbanak cyber heist gang members arrested. Computer Weekly. Retrieved from https://www.computerweekly.com/news/252446153/ThreeCarbanak-cyber-heist-gang-members-arrested Atherton, K. D. (2018, December 26). What does 2019 hold for Russia’s drones? C4ISRNet. Retrieved from https://www.c4isrnet.com/newsletters/unmannedsystems/2018/12/26/what-does-2019-hold-for-russias-drones/ Atlantic Council. (2019, April 3). About the council: Working together to secure. Renewing the Atlantic community for global challenges. Retrieved from https://www.atlanticcouncil.org/about Bartosh, A. A. (2018, October 4). Hybrid threats on the OSCE Agenda. Independent Military Review. (Независимое Военное Обзрение). Retreived from http://nvo.ng.ru/realty/201810-04/1_1016_osce.html BBC News. (2018, November 12). Russia suspected of jamming GPS signal in Findland. Retrieved from https://www.bbc.com/news/world-europe-46178940 91 Besemeres. J. (2016). Russian disinformation and Western misconceptions. In A difficult neighborhood: Essays on Russia and east-central Europe since World War II. Sidney, Australia: Australian National Press. Retrieved from https://www.jstor.org/stable/j.ctt1rqc96p.33 Black Hat Europe. (2017). The 2017 Black Hat Europe attendee survey: The cyberthreat in Europe. Retrieved from https://www.blackhat.com/docs/eu-17/Black-Hat-AttendeeSurvey.pdf Bartles, C. K. (2016). Getting Gerasimov right. Military Review, 96(1), p. 30.38. Retrieved from http://usacac.army.mil/CAC2/MilitaryReview/Archives/English/MilitaryReview_201602 28_art009.pdf Bartles, C. K. (2017). Recommendations for intelligence staffs concerning Russian new generation warfare. Army Press. Retrieved from https://www.armyupress.army.mil/Portals/7/Hot%20Spots/Documents/Russia/Bartlesrussian.pdf Berg-Knutsen, E. (2016). From tactical champions to grand strategy enablers: The future of small-nation SOF in counter-hybrid warfare. In Skinner, E. (Eds.) Countering hybrid warfare: The best uses of SOF in a pre-Article V scenario. CTX, 6(4), 61-68. Retrieved from https://www.ffi.no/no/Publikasjoner/Documents/CTX_Countering Hybrid Warfare. The best use of SOF in a pre-article V Scenario.pdf Binnendijk, H. (2016). The role of NATO joint air power in deterrence and collective defense. Joint Air Power Competence Centre, presented in Joint Air Power following the 2016 Warsaw Summit – Urgent Priorities, p. 56. Retrieved from https://www.japcc.org/wpcontent/uploads/Joint_Air_Power_Following-_Warsaw_-Summit.pdf 92 Blank, S. (2015, February 4). A military assessment of the Russian war in Ukraine. Testimony presented to the Senate Foreign Relations Committee, Subcommittee on Europe and Regional Security Cooperation. Retrieved from https://www.foreign.senate.gov/imo/media/doc/Blank%20Testimony.pdf Blinde, L. (2016a). USSOCOM releases industry collaboration tool, announces upcoming event. Intelligence Community News. Retrieved from https://intelligencecommunitynews.com/ussocom-releases-industry-collaboration-toolannounces-upcoming-event/ Blinde, L. (2016b). DIA to hold meeting on upcoming National Media Exploitation Center acquisition. Intelligence Community News. Retrieved from https://intelligencecommunitynews.com/dia-to-hold-meeting-on-upcoming-nationalmedia-exploitation-center-acquisition/ Borgatti, S. P., Everett, M. G., & Johnson, J. C. (2013). Analyzing social networks. London, UK: SAGE. Borio, D., O’Driscoll, C., & Fortuny, J. (2013). Jammer impact on Galileo and GPS receivers. EC Joint Research Centre, Institute for the Protection and Security of the Citizen, Ispra, Italy. Retrieved from https://www.researchgate.net/publication/261488817_Jammer_impact_on_Galileo_and_ GPS_receivers Boston, S., & Massicot, D. (2017). The Russian way of warfare: A primer. RAND Corporation. Retrieved from https://www.rand.org/content/dam/rand/pubs/perspectives/PE200/PE231/RAND_PE231. pdf 93 Boyes, H., Hallaq, B., Cunningham, J., & Watson, T. (2016). The industrial internet of things (IIoT): An analysis framework. Computers in Industry, 101, March. Retrieved from https://ac-els-cdn-com.ezproxy.utica.edu/S0166361517307285/1-s2.0S0166361517307285-main.pdf?_tid=973ed476-d613-4fd5-a9a945d55463e7ab&acdnat=1552334438_3b8e2caa55295efc7bc3d5811479e549 Braccini, C., Väisänen, T., Sadloň, M., Bahşi, H., Panico, A., van der Meij, K., & Huis in ‘t veld, M., 2016). Battlefield digital forensics: Digital intelligence and evidence collection in special operations. NATO Cooperative Cyber Defence Centre of Excellence (NATO CCDCOE). Retrieved from http://docplayer.net/32774382-Battlefield-digitalforensics.html Bradshaw S., & Howard, P. N. (2017). Troops, trolls and troublemakers: A global inventory of organized social media manipulation. Computational Propaganda Research Project, 12, University of Oxford. Retrieved from https://ora.ox.ac.uk/objects/uuid:cef7e8d9-27bf4ea5-9fd6-855209b3e1f6/download_file?safe_filename=Troops-Trolls-andTroublemakers.pdf&file_format=application%2Fpdf&type_of_work=Report Breede, C. H. (2018). Special (peace) operations: Optimizing SOF for UN missions. International Journal, 73(2), 221-240. Retrieved from https://journals-sagepubcom.ezproxy.utica.edu/doi/pdf/10.1177/0020702018787633 Brenner, S., & Schwerha, J. (2002). Transnational evidence gathering and local prosecution of international cybercrime. Journal of Computer & Information Law, 20(1), 243-258. Retrieved from http://heinonline.org.ezproxy.utica.edu/HOL/Page?handle=hein.journals/jmjcila20& div=17&g_sent=1&casa_token=&collection=journals# 94 Broadhurst, R., Grabosky, P., Alazab, M., & Chon, S. (2014). Organizations and cyber crime: An analysis of the nature of groups engaged in cyber crime. International Journal of Cyber Criminology, 8(1), 1-20. Retrieved from https://www.cybercrimejournal.com/broadhurstetalijcc2014vol8issue1.pdf Brown, J. (2017, November 17). Lithuania caught up in spying, illicit border trade. DW. https://www.dw.com/en/lithuania-caught-up-in-spying-illicit-border-trade/a-36792909 Brzozowski, A. (2019, February 14). NATO in search of options to prepare for post-INF world. Euractiv.com. Retrieved from https://www.euractiv.com/section/defence-andsecurity/news/nato-in-search-of-options-to-prepare-for-post-inf-world/ Bugajski, J. (2016). Only NATO can defend Europe. European View, 15(1), 27-35. Retrieved from https://journals-sagepub-com.ezproxy.utica.edu/doi/pdf/10.1007/s12290-016-03839 Burakova, Y., Hass, B., Millar, L., & Weimerskirch, A. (2016). Truck hacking: An experimental analysis of the SAE J1939 Standard. Presented at the 10th USENIX Conference on Offensive Technologies, Austin, TX., 2016. Retrieved from https://www.usenix.org/conference/woot16/workshopprogram/presentation/burakova Carlin, J. P., Ledgett, R., Miller Jr., J. N., & Lewis, J. A. (2018). “Responding to Russia: Deterring Russian cyber and grey zone activities”. Panel discussion. Center for Strategic and International Studies. Retrieved from https://csis-prod.s3.amazonaws.com/s3fspublic/publication/180316_Responding_Russia_Grey_zone.pdf Carstens, R. (2017). Explaining Russia through Putin: A review of the new tsar by Steven Lee Myers. Journal of International Affairs, 72(1). Retrieved from 95 https://jia.sipa.columbia.edu/explaining-russia-through-putin-review-new-tsar-steven-leemyers Center for the Study of the Presidency & Congress. (2014). Securing the U.S. electrical grid: Understanding the threats to the most critical of critical infrastructure, while securing a changing grid. Retrieved from https://www.thepresidency.org/sites/default/files/Final%20Grid%20Report_0.pdf Cerulus, L. (2019, February 14). How Ukraine became a test bed for cyber weaponry. Politico. https://www.politico.eu/article/ukraine-cyber-war-frontline-russia-malware-attacks/ Chatfield, A. T., & Reddick, C. G. (2018). A framework for Internet of Things-enabled smart government: A case of IoT cybersecurity policies and use cases in U.S. federal government. Government Information Quarterly, Winter 2018. Retrieved from Chen, A. (2015, June 2). The agency. The New York Times. Retrieved from https://www.nytimes.com/2015/06/07/magazine/the-agency.html Chen, Y., Lu, Z., Xiong, H., & Xu, W. (2018). Privacy-preserving data aggregation protocol for fog computing-assisted vehicle-to-infrastructure scenario. Security and Communication Networks, April 2018. Retrieved from http://web.a.ebscohost.com.ezproxy.utica.edu/ehost/pdfviewer/pdfviewer?vid=0&sid=20 d2cff6-fa61-450a-acc7-65b8bd89fc6e%40sdc-v-sessmgr06 Cimpanu, C. (2017, July 11). Prolific Russian hacker gets nine years in prison. Bleeping Computer. Retrieved from https://www.bleepingcomputer.com/news/security/prolificrussian-hacker-gets-nine-years-in-prison/ Cimpanu, C. (2018a, March 20). AMD confirms RyzenFall, MasterKey, Fallout, and Chimera vulnerabilities. Bleeping Computer. Retrieved from 96 https://www.bleepingcomputer.com/news/hardware/amd-confirms-ryzenfall-masterkeyfallout-and-chimera-vulnerabilities/ Cimpanu, C. (2018b, March 26). Leader of Carbanak (Cobalt) hacker group who stole over 1 billion arrested in Spain. Bleeping Computer. Retrieved from https://www.bleepingcomputer.com/news/security/leader-of-carbanak-cobalt-hackergroup-who-stole-over-1bil-arrested-in-spain/ Ciulla, V. (2016, September 2). Electronic control unit - ECU. ThoughtCo. Retrieved from https://www.thoughtco.com/electronic-control-unit-ecu-4083663 Cisco (2019, February 18). Cisco visual networking index: Global mobile data traffic forecast update, 2017-2022. Retrieved from https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networkingindex-vni/white-paper-c11-738429.pdf Ciufo, C. A. (2018, October 2). How "near-peer" adversaries are changing command posts and battlefield network requirements. Military Embedded Systems. Retrieved from http://milembedded.com/articles/how-posts-battlefield-network-requirements/ Cleveland, C. T., Linder, J. B., & Dempsey, R. (2016). Special operations doctrine: Is it needed? PRISM, 6(3), 5-19. Retrieved from https://www.hsdl.org/?view&did=797974 Conley, H., Mina, J., Stafanov, R., & Vladimirov, M. (2016). The Kremlin’s playbook: Understanding Russian influence in central and eastern Europe. New York, NY: Rowman & Littlefield. Retrieved from https://www.csis.org/ analysis/kremlin-playbook Connell, M., & Vogler, S. (2017). Russia’s approach to cyber warfare. CNA. Retrieved from https://www.cna.org/cna_files/pdf/DOP-2016-U-014231-1Rev.pdf 97 Convention on Cybercrime. (2001, November 23). Council of Europe Treaty Series, Budapest, 185. Retrieved from https://www.coe.int/en/web/conventions/full-list//conventions/treaty/185 Cranny-Evans, S., Cazalet, M., & Foss, C. F. (2018). The czar of battle: Russian artillery use in Ukraine portends advances. Janes’s by HIS Markit. Retrieved from https://www.janes.com/images/assets/111/80111/The_Czar_of_battle_Russian_artillery_ use_in_Ukraine_portends_advances.pdf Crist, K. R. (2017). Utilization of location information on digital media devices (Master’s thesis). Utica College. Retrieved from https://search-proquestcom.ezproxy.utica.edu/pqdtlocal1008803/docview/1897539606/fulltextPDF/24C1B0308 7AB45DAPQ/3?accountid=28902 Cronk, T. M. (2018, May 4). Near-peer adversaries work to surpass U.S. in technology, official says. U.S. Department of Defense. Retrieved from https://dod.defense.gov/News/Article/Article/1512901/near-peer-adversaries-work-tosurpass-us-in-technology-official-says/ Crowther, G. A. (2017). The cyber domain. The Cyber Defense Review, 2(3), pp. 63-78. Retrieved from https://www.jstor.org/stable/10.2307/26267386 Cullen, P. (2018). Hybrid threats as a new ‘wicked problem’ for early warning. Strategic Analysis, May. Retrieved from https://www.hybridcoe.fi/wpcontent/uploads/2018/06/Strategic-Analysis-2018-5-Cullen.pdf Cullen, P. J., & Reichborn-Kjennerud, E. (2017). MCDC countering hybrid warfare project: Understanding hybrid warfare. Multinational Capability Development Campaign. Retrieved from 98 https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_ data/file/647776/dar_mcdc_hybrid_warfare.pdf Combating Terrorism Exchange. (2016). Countering hybrid warfare: The best uses of SOF in a pre-Article V scenario. Skinner, E. (Eds.) Retrieved from https://www.ffi.no/no/Publikasjoner/Documents/CTX_Countering Hybrid Warfare. The best use of SOF in a pre-article V Scenario.pdf Czuperski, M., Herbst, J., Higgins, E., Polyakova, A., & Wilson, D. (2015). Hiding in plain sight: Putin’s war in Ukraine. Atlantic Council. Retrieved from https://www.atlanticcouncil.org/publications/reports/hiding-in-plain-sight-putin-s-war-inukraine-and-boris-nemtsov-s-putin-war Danylyuk, O. (2016). Russian aggression toward Ukraine: A long-term example of hybrid warfare. In Skinner, E. (Eds.) Countering hybrid warfare: The best uses of SOF in a preArticle V scenario. CTX, 6(4), Retrieved from https://www.ffi.no/no/Publikasjoner/Documents/CTX_Countering Hybrid Warfare. The best use of SOF in a pre-article V Scenario.pdf Daugherty, P., Banerjee, P., Nejm, W., & Alter, A. E. (2015). Driving unconventional growth through the Industrial Internet of Things. Accenture Technology. Retrieved from https://www.accenture.com/au-en/_acnmedia/Accenture/next-gen/reassemblingindustry/pdf/Accenture-Driving-Unconventional-Growth-through-IIoT.pdf Deep, A. (2015, March 2). Hybrid War Old Concept New Techniques. Small Wars Journal. Retrieved from http://smallwarsjournal.com/jrnl/art/hybrid-war-old-concept-newtechniques 99 Defense Science Board Task Force on Cyber Deterrence. (2017). Final report of the Defense Science Board (DSB) Task Force on cyber deterrence. U.S. Department of Defense. Retrieved from https://www.acq.osd.mil/dsb/reports/2010s/DSBCyberDeterrenceReport_02-28-17_Final.pdf DeGrazia, M. (2018). Finding and decoding malicious powershell scripts [Web video]. Presented at SANS DFIR Summit 2018. Retrieved from https://www.youtube.com/watch?v=JWC7fzhvAY8 Deloitte. (2017). Defense policy and the Internet of Things disrupting global cyber defenses. Deloitte Japan. Retrieved from https://www2.deloitte.com/jp/en/pages/publicsector/articles/gv/defense-policy-and-the-internet-of-things.html DeTrinis, S. E. (2017). Cold war redux: Shaping the Arctic as strategic maneuver space. In M. R. Slater, M. Purcell, & A. M. Del Gaudio (Eds.), Considering Russia: Emergence of a Near Peer Competitor (50-53). Diaz, V., & Raiu, C. (2018, December 5). APT review of the year: What the world’s advanced threat actors got up to in 2018. Kaspersky Lab. Retrieved from https://securelist.com/aptreview-of-the-year/89117/ Digital Forensic Research Lab (DFRLab) (2019, April 3). Our mission. Atlantic Council. Retrieved from https://www.digitalsherlocks.org/about Dobromirov, V., Dotsenko, S., Verstov, V., & Volkov, S. (2017). Methods of examining vehicle electronic systems in the course of automotive forensic expert examinations. Presented at the 12th International Conference "Organization and Traffic Safety Management in Large Cities", SPbOTSIC-2016, 28-30 September 2016, St. Petersburg, Russia. Retrieved from http://dx.doi.org/10.1016/j.trpro.2017.01.037 100 Dorschner, J., & White, A. (2014). Quiet professionals: NATO special operations comes of age. HIS Jane’s Defence Weekly. Retrieved from https://www.janes.com/images/assets/968/51968/NATO_special_operations_comes_of_a ge.pdf Ellis, D. C., Black, C. N., & Nobles, M. A. (2017). Thinking dangerously: Imagining United Special Operations Command in the post-ct world. PRISM, 6(3), 111-129. Retrieved from https://apps.dtic.mil/dtic/tr/fulltext/u2/1042559.pdf Erlingsson, E. (2018, October 19). A credible transatlantic bond: Trident Juncture and NATO capabilities. NATO Review Magazine. Retrieved from https://www.nato.int/docu/review/2018/also-in-2018/a-credible-transatlantic-bondtrident-juncture-and-nato-capabilities-military-exercise/EN/index.htm European Centre of Excellence for Countering Hybrid Threats. (2019, March 24). What is Hybrid COE? Retrieved from https://www.hybridcoe.fi/what-is-hybridcoe/ European Commission. (2018). Energy security strategy. Retrieved from https://ec.europa.eu/energy/en/topics/energy-strategy-and-energy-union/energy-securitystrategy European Defense Agency. (2014a, July 29). Latest news counter-IED lab arrives in the Netherlands. Retrieved from https://www.eda.europa.eu/info-hub/press-centre/latestnews/2014/07/29/counter-ied-lab-arrives-in-the-netherlands European Defence Agency. (2014b, November 4). Latest news: New facility to help in fight against IEDs opens in the Netherlands. Retrieved from https://www.eda.europa.eu/infohub/press-centre/latest-news/2014/11/04/new-facility-to-help-in-fight-against-ieds-opensin-the-netherlands 101 European Defence Agency. (2014c, November 6). Joint Deployable Exploitation and Analysis Laboratory (JDEAL). Retrieved from https://www.eda.europa.eu/what-wedo/activities/activities-search/joint-deployable-exploitation-and-analysis-laboratory(jdeal) European Space Agency. (2019, March 28). File: GNSS navigational frequency bands.png. Retrieved from https://gssc.esa.int/navipedia/index.php/File:GNSS_navigational_frequency_bands.png European Values. (2017). Prague Declaration on seven urgent steps proposed by Western security experts: “How the democratic West should stop Putin”. Retrieved from https://www.europeanvalues.net/declaration/ Europol. (2018). Internet organized crime threat assessment (IOCTA) 2018. Retrieved from https://www.europol.europa.eu/activities-services/main-reports/internet-organisedcrime-threat-assessment-iocta-2018 Fábián, S. (2016). To change or not to change? In Countering hybrid warfare: The best uses of SOF in a pre-Article V scenario, CTX, 6(4), 69-74. Retrieved from https://www.ffi.no/no/Publikasjoner/Documents/CTX_Countering Hybrid Warfare. The best use of SOF in a pre-article V Scenario.pdf Farley, R. (2018, November 26). Meet the 5 Russian weapons of war Ukraine should fear. National Interest. Retrieved from https://nationalinterest.org/blog/buzz/meet-5-russianweapons-war-ukraine-should-fear-37112 Fazzini, K., & Macias, A. (2019, March 27). Peter Thiel’s company Palantir just won a major Pentagon contract beating out traditional military vendors. CNBC. Retrieved from 102 https://www.cnbc.com/2019/03/27/palantir-in-multi-million-dollar-pentagon-deal-ipo-onhorizon.html Feltey, T., & Rae, C. (2018). Military deception and reverse intelligence preparation of the battlefield: How staff integration creates advantages for the Brigade Combat Team Commander. eARMOR, Fall 2018. Retrieved from https://www.benning.army.mil/ARMOR/eARMOR/content/issues/2018/Fall/4RaeFeltey18.pdf FireEye. (2015). Cyber threats to the Nordic region. Retrieved from https://www.fireeye.com/content/dam/fireeye-www/global/en/current-threats/pdfs/rptnordic-threat-landscape.pdf FireEye. (2016). The cost of no context: The value of true cyber threat intelligence. Retrieved from https://www.fireeye.com/solutions/cyber-threat-intelligence/wp-true-value-cti.html Fisher, D. (2018, October 4). Russian APTs Turla and Sofancy sharing code and targets. Decipher. Retrieved from https://duo.com/decipher/russian-apts-turla-and-sofacysharing-code-and-targets Fisher, D. (2019, February 12). APT groups moving down the supply chain. Decipher. Retrieved from https://duo.com/decipher/apt-groups-moving-down-the-supply-chain Floyd Jr., G. (2018). Attribution and operational art: Implications for competing in time. Strategic Studies Quarterly, 12(2), 17-55. Fox, A. C. (2017). Battle of Debal’tseve: The conventional line of effort in Russia’s hybrid war in Ukraine. eARMOR, Fall 2017. Retrieved from https://www.benning.army.mil/ARMOR/eARMOR/content/issues/2017/Winter/1Fox17.p df 103 Freedom House. (2017). Freedom on the net 2017. Retrieved from https://freedomhouse.org/sites/default/files/FOTN_2017_Final.pdf Freytag von Loringhoven, A. (2017, Augutst 9). Adapting NATO intelligence in support of “One NATO”. NATO Review Magazine. Retrieved from https://www.nato.int/docu/review/2017/Also-in-2017/adapting-nato-intelligence-insupport-of-one-nato-security-militaryterrorism/EN/index.htm?fbclid=IwAR3BHvdEwyXsAvKBGPN3PJ2oHdmB2mpnuTPONj63453YWLaJKR3mMlW9rU Fryc, M. (2016). From Wales to Warsaw and beyond: NATO’s strategic adaptation to the Russian resurgence on Europe’s eastern flank. Connections, 15(4), 45-65. Retrieved from https://www.jstor.org/stable/10.2307/26542702 Gady, F. (2015, March 3). Russia tops China as principal cyber threat to US. The Diplomat. Retrieved from https://thediplomat.com/2015/03/russia-tops-china-as-principal-cyberthreat-to-us/ Galer, A. (2018, September 14). Russian REX-1 counter-UAV system breaks cover on exercise. Jane’s International Defence Review. Retrieved from https://www.janes.com/article/82990/russian-rex-1-counter-uav-system-breaks-cover-onexercise Garamone, J. (2015, August 17). Dempsey: U.S. forces must adapt to deal with near-peer competitors. U.S. Department of Defense. Retrieved from https://dod.defense.gov/News/Article/Article/613843/dempsey-us-forces-must-adapt-todeal-with-near-peer-competitors/ 104 Garcia, P. (2018, November 1). Don’t trust your hardware: Why security vulnerabilities affect us all. The Conversation. Retrieved from http://theconversation.com/dont-trust-yourhardware-why-security-vulnerabilities-affect-us-all-105773 Gardner, J. V. (2017). A duty to share: The opportunities and obstacles of federal counterterrorism intelligence sharing with nonfederal fusion centers [Doctoral dissertation]. Walden University. Retrieved from https://scholarworks.waldenu.edu/cgi/viewcontent.cgi?article=4873&context=dissertation s Geers, K. (2009). The cyber threat to national critical infrastructures: Beyond theory. Information Security Journal: A Global Perspective, 18, p. 1-7. Geers, K., Kindlund, D., Moran, N., & Rachwald, R. (2015). World war C: Understanding nation-state motives behind today’s advanced cyber attacks. FireEye. Retrieved from https://www.fireeye.com/content/dam/fireeye-www/global/en/currentthreats/pdfs/fireeye-wwc-report.pdf Gerasimov, V. V. (2013). The value of science in the foresight: New challenges demand rethinking the forms and methods of carrying out combat operations. Military-Industrial Courier, 8(476), 23-29. Retrieved from https://usacac.army.mil/CAC2/MilitaryReview/Archives/English/MilitaryReview_20160 228_art008.pdf Giles, K. (2016). Handbook of Russian Information Warfare. NATO Defense College “NDC Fellowship Monograph Series”. Retrieved from https://www.researchgate.net/publication/313423985_Handbook_of_Russian_Informatio n_Warfare 105 Giles, K., & Hagestad II, W. (2013). Divided by a common language: Cyber definitions in Russian, Chinese, and English. Presented at the 5th International Conference on Cyber Conflict, Tallinn, Estonia. Podins, K, Stinissen, J, & Maybaum, M. (Eds.). Retrieved from https://www.researchgate.net/publication/261300676 Gillware. (2018), March 27). Ryzenfall: Another bombshell AMD vulnerability. Retrieved from https://www.gillware.com/data-recovery-lab/ryzenfall-amd-vulnerability/ Glatz, R., & Zapfe, M. (2016). NATO defence planning between Wales and Warsaw. Politicomilitary challenges of a credible assurance against Russia. German Institute for International and Security Affairs. Retrieved from https://www.swpberlin.org/fileadmin/contents/products/comments/2016C05_glt_Zapfe.pdf Gramer, R. (2018, February 20). If America is first, is NATO second? An interview with NATO Secretary-General Jens Stoltenberg. Foreign Policy. Retrieved from https://foreignpolicy.com/2018/02/20/nato-secretary-general-jens-stoltenberg/ Grau, L., & Bartles, C. K. (2016). The Russian way of war: Force structure, tactics, and modernization of the Russian ground forces. Fort Leavenworth: Foreign Military Studies Office. Retrieved from https://www.armyupress.army.mil/Portals/7/Hot%20Spots/Documents/Russia/2017-07The-Russian-Way-of-War-Grau-Bartles.pdf Greaver, B., Raabe, L., Fox, W. P., & Burks, R. E. (2018). CARVER 2.0: integrating the Analytical Hierarchy Process’s multi-attribute decision-making weighting scheme for a center of gravity vulnerability analysis for US Special Operations Forces. Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, 15(1), 111120. 106 Greenberg, A. (2018, August 22). The untold story of NotPetya, devastating cyberattack in history. Wired. Retrieved from https://www.wired.com/story/notpetya-cyberattackukraine-russia-code-crashed-the-world/ Grigas, A. (2016). Beyond Crimea: The new Russian empire. New Haven, CT: Yale University Press. Haddick, R. (2016). Improving the sustainment of SOF distributed operations in access-denied environments. Tampa, FL: Joint Special Operations University. Retrieved from https://www.hsdl.org/?abstract&did=789778 Hale, K. A. (2012). Expanding the use of time/frequency difference of arrival geolocation in the department of defense [Dissertation]. Pardee RAND Graduate School. Retrieved from http://www.rand.org/content/dam/rand/pubs/rgs_dissertations/2012/RAND_RGSD3 08.pdf Harper, J. (2017, May 19). SOCOM seeks quicker ways of exploiting captured cellphone data. National Defense. Retrieved from http://www.nationaldefensemagazine.org/articles/2017/5/19/socom-seeks-better-ways-ofexploiting-cellphone-and-computer-data Harvey, J. (2017, May 4). The shadowy-and vital-role attribution plays in cybersecurity. Accenture. Retrieved from https://www.accenture.com/us-en/blogs/blogs-shadowy-vitalrole-attribution-cybersecurity Haybeck, M., & Harrison, R. (2016). The Russian way of war: Force structure, tactics, and modernization of the Russian ground forces. The Journal of Military History, 66(3), 1402. Retrieved from https://www.armyupress.army.mil/Portals/7/Hot Spots/Documents/Russia/2017-07-The-Russian-Way-of-War-Grau-Bartles.pdf 107 Herion, O. R. (2012). Expeditionary forensic support to Joint Force Commanders: What changes or considerations are warranted? Masters Thesis. Marine Corps University. Retrieved from https://apps.dtic.mil/dtic/tr/fulltext/u2/a601187.pdf Heue, R. (2018, June 4). GNSS jamming and spoofing: Hazard or hype. Space of Innovation. Retrieved from https://www.space-of-innovation.com/gnss-jamming-and-spoofinghazard-or-hype/ Hoffman, F. G (2007). Conflict in the 21st Century: The Rise of Hybrid War, Arlington, VA: Potomac Institute for Policy Studies. Retrieved from http://www.potomacinstitute.org/images/stories/publications/potomac_hybridwar_0108.p df Hoffman, F. G. (2016). The contemporary spectrum of conflict: Protracted, gray zone, ambiguous, and hybrid modes of war. The Heritage Foundation. Retrieved from https://s3.amazonaws.com/ims2016/PDF/2016_Index_of_US_Military_Strength_ESSAYS_HOFFMAN.pdf Hoffman, F. G. (2018). Examining complex forms of conflict: Gray zone and hybrid challenges. PRISM, 7(4), 30-47. Retrieved from https://www-jstororg.ezproxy.utica.edu/stable/pdf/26542705.pdf Houser, N., & Johnson, T. (2017, October 2). Data-driven deployments: How analytics can transform military positioning. Deloitte Insights. Retrieved from https://www2.deloitte.com/insights/us/en/industry/public-sector/mission-analyticsmilitary-deployments.html Howard, P. N., Ganesh, B., & Liotsiou, D. (2018). The IRA, social media and political polarization in the United States, 2012-2018. Computational Propaganda Research 108 Project. University of Oxford. Retrieved from https://comprop.oii.ox.ac.uk/wpcontent/uploads/sites/93/2018/12/IRA-Report-2018.pdf HTBIS. (2019, February 2). Russians are buying 9 properties per day in Spain. Where do they buy? What do they like? Retrieved from https://howtobuyinspain.com/en/russiansbuying-property-spain/ Hurd, H., & Chachko, E. (2018, October 25). U.S. withdraw from the INF treaty: The facts and the law. Lawfare. Retrieved from https://www.lawfareblog.com/us-withdrawal-inf-treatyfacts-and-law Intelligence and National Security Alliance (INSA). (2013). Operational levels of cyber intelligence. INSA Cyber Intelligence Task Force. Retrieved from https://www.insaonline.org/wpcontent/uploads/2017/04/INSA_OperCyberIntelligence_WP.pdf INSA. (2014a). Strategic cyber intelligence. INSA Cyber Intelligence Task Force. Retrieved from https://www.insaonline.org/wpcontent/uploads/2017/04/INSA_StrategicCyberIntel_WP.pdf INSA. (2014b). Operational cyber intelligence. INSA Cyber Intelligence Task Force. Retrieved from https://www.insaonline.org/wpcontent/uploads/2017/04/INSA_WP_Op_Cyber_FIN.pdf INSA. (2015). Tactical cyber intelligence. INSA Cyber Intelligence Task Force. Retrieved from https://www.insaonline.org/tactical-cyber-intelligence/ International Association of Crime Analysts. (2018). Social network analysis for law enforcement. White Paper for the Standards, Methods, & Technology Committee (2018- 109 02). Retrieved from https://crimegunintelcenters.org/wpcontent/uploads/2018/07/iacawp_2018_02_social_network_analysis.pdf James, B. W. (2016). Sharpening the Spear of NATO SOF: Deterring Russian hybrid aggression through network targeting. CTX, 6(4), 75-81. Retrieved from https://www.ffi.no/no/Publikasjoner/Documents/CTX_Countering Hybrid Warfare. The best use of SOF in a pre-article V Scenario.pdf Janda, J. (2018). How to boost the Western response to Russian hostile influence operations. European View, 17(2), 181-188. Retrieved https://journals-sagepubcom.ezproxy.utica.edu/doi/pdf/10.1177/1781685818803524 Juurvee, I. (2018). The resurrection of ‘active measures’: Intelligence services as a part of Russia’s influencing toolbox. Strategic Analysis, April. Hybrid CoE. Retrieved from https://www.hybridcoe.fi/wp-content/uploads/2018/05/Strategic-Analysis-2018-4Juurvee.pdf Kadushin, C. (2012). Understanding social networks: Theories, concepts, and findings. New York, NY: Oxford University Press. Kaspersky Lab. (2015). Carbanak APT: The great bank robbery (Version 2.1). Retrieved from https://media.kasperskycontenthub.com/wpcontent/uploads/sites/43/2018/03/08064518/Carbanak_APT_eng.pdf Kaspersky Lab ISC CERT. (2019, January 24). GreyEnergy’s overlap with Zebrocy. Retrieved from https://securelist.com/greyenergys-overlap-with-zebrocy/89506/ Keating, D. (2018, March 22). European power plants brace for Russian hack attacks. Forbes. Retrieved from https://www.forbes.com/sites/davekeating/2018/03/22/european-powerplants-brace-for-russian-hack-attacks/#21bece4e7226 110 Kiesewetter, R., & Zielke, I. (2016). Permanent NATO deployment is not the answer to European security. European View, 15(1), 37-45. Retrieved from https://journalssagepub-com.ezproxy.utica.edu/doi/pdf/10.1007/s12290-016-0392-8 Klintsevich, F. (2017, September 26). Russia alarmed by U.S. Army warfare ‘handbook’. The Moscow Times. Retrieved from https://www.themoscowtimes.com/2017/09/26/russiaalarmed-by-us-army-warfare-handbook-a59075 Kofman, M., Migacheva, K., Nichiporuk, B., Radin, A., Tkacheva, O., & Oberholtzer, J. (2017). Lessons from Russia’s operations in Crimea and Eastern Ukraine. RAND Corporation. Retrieved from https://www.rand.org/content/dam/rand/pubs/research_reports/RR1400/RR1498/RA ND_RR1498.pdf Kofman, M., & Rojansky, M. (2015). A closer look at Russia’s “Hybrid War”. Kennan Cable, 7. Retrieved from https://www.files.ethz.ch/isn/190090/5-kennan%20cablerojansky%20kofman.pdf Kollars, N. A., & Petersen, M. B. (2018). Feed the bears, starve the trolls: Demystifying Russia’s cybered information confrontation strategy. Presented at CyCon U.S., 2018. Army Cyber Institute, West Point. Retrieved from https://www.hsdl.org/?abstract&did=818913 Korkisch, F. W. (2010). NATO gets better intelligence: New challenges require new answers to satisfy intelligence needs for headquarters and deployed/employed forces. Center for Foreign and Defense Policy, Vienna, Austria. Kramer, F. D., Binnendijk, H., & Speranza, L. M. (2018). NATO priorities: After the Brussels summit. Atlantic Council. Retrieved from 111 https://www.atlanticcouncil.org/images/publications/NATO-Priorities-After-theBrussels-Summit.pdf Krebs on Security. (2017, April 24). The backstory behind Carder kingpin Roman Seleznev’s record 27 year prison sentence. Krebsonsecurity.com. Retrieved from https://krebsonsecurity.com/2017/04/the-backstory-behind-carder-kingpin-romanseleznevs-record-27-year-prison-sentence/ Krott, Livingston, & Morales (2012). Development of a rapidly deployable Special Operations Component Command (SOCC) core concept for the North Atlantic Treaty Organization (NATO Special Operations Headquarters (NSHQ) (Masters thesis). U.S. Naval Postgraduate School, Monterey, California. Received from https://calhoun.nps.edu/bitstream/handle/10945/10635/11Dec%255FKrott%255FMBA.p df?sequence=1&isAllowed=y Kyle, J. (2019). Contextualizing Russia and the Baltic States. Foreign Policy Research Institute, 63(1), 104-115. Retrieved from https://doi.org/10.1016/j.orbis.2018.12.004 LabSat. (2019). GNSS frequency guide – Labsat 3 wideband. Retrieved from https://www.labsat.co.uk/index.php/en/applications/labsat-frequency-guide Larrabee, F. S., Pezard, S., Radin, A., Chandler, N., Crane, K., & Szayna, T. S. (2017). Russia and the West after the Ukrainian crisis: European vulnerabilities to Russian pressures. RAND Corporation. Retrieved from https://www.rand.org/content/dam/rand/pubs/research_reports/RR1300/RR1305/RAND_ RR1305.pdf Lee, R. M., Assante, M. J., & Conway, T. (2016). Analysis of the cyber attack on the Ukrainian power grid. SANS Industrial Control Systems & Electricity Information Sharing and 112 Analysis Center. Retrieved from https://ics.sans.org/media/EISAC_SANS_Ukraine_DUC_5.pdf Le-Khac, N., Jacobs, D., Nijhoff, J., Bertens, K., & Choo, K. R. (2018). Smart vehicle forensics: Challenges and case study. Future Generation Computer Systems, May 2018. Retrieved from https://doi.org/10.1016/j.future.2018.05.081 Lilly, A. (2017). IMSI catchers: hacking mobile communications. Network Security, 2, 5-7. Retrieved from https://ac-els-cdn-com.ezproxy.utica.edu/S1353485817300144/1-s2.0S1353485817300144-main.pdf?_tid=c9fb905b-01ba-46d3-a832833bbdef838c&acdnat=1552334314_d014c8dd52ee6ecc7cb6b34997112f7f Limnéll, J. (2018). Countering hybrid threats: Role of private sector increasingly important. Shared responsibility needed. Strategic Analysis, March. Hybrid CoE. Retrieved from https://www.hybridcoe.fi/wp-content/uploads/2018/03/Strategic-Analysis-2018-3Limnell.pdf Lin, K., Xia, F., Li, S., Wang, D., & Humar, I. (2019). Emotion-aware system design for the battlefield environment. Information Fusion, 47, 102-110. Retrieved from https://ac-elscdn-com.ezproxy.utica.edu/S1566253517306991/1-s2.0-S1566253517306991main.pdf?_tid=5efc080c-13bc-4f0b-90244aad15ebdcb3&acdnat=1552321930_d38170f02108c16e96ada7dd532970fd Linder, A. (2018). Russian private military companies in Syria and beyond. Center for Strategic and International Studies, Retrieved from https://csis-prod.s3.amazonaws.com/s3fspublic/181017_RussianPrivateMilitary.pdf?EdPhoXOlhiGQM2BcYZyQiug3_aJem5nM 113 Linthicum, D. (2019, March 17). Edge computing vs. fog computing: Definitions and enterprise uses. Cisco. Retrieved from https://www.cisco.com/c/en/us/solutions/enterprisenetworks/edge-computing.html Llewellyn, M. (2017). DJI Phantom 3-drone forensic data exploration. Edith Cowan University. Perth, Australia. Retrieved from https://www.researchgate.net/publication/329879540 DJI Lockheed Martin. (2019, April 5). The Cyber Kill Chain. Retrieved from https://www.lockheedmartin.com/en-us/capabilities/cyber/cyber-kill-chain.html Long, A. (2014). NATO special operations: Promise and problem. ORBIS, 58(4), 540-551. Retrieved from http://dx.doi.org/10.1016/j.orbis.2014.08.006 Lunan, M., Moore, J., Moore J., ter Horst, W. (2018). New doctrinal concepts: Biometrics. Three Swords Magazine, 33. Retrieved from http://www.jwc.nato.int/images/stories/threeswords/Biometrics_2018.pdf Lusthaus, J. (2013). How organised is organised cyber crime? Global Crime, 14(1), 52–60. Retrieved from https://www.tandfonline.com/doi/abs/10.1080/17440572.2012.759508?journalCode=fglc 20 Lysenko, V., & Brooks, C. (2018). Russian information troops, disinformation, and democracy. First Monday, 23(5). Retrieved from https://journals.uic.edu/ojs/index.php/fm/article/view/8176/7201 Mancini L.V., Monti A., Panico A. (2017) SOF on Trial. The technical and legal value of battlefield digital forensics in court. In: Shyamasundar R., Singh V., Vaidya J. (Eds) Information Systems Security. ICISS 2017. Lecture Notes in Computer Science, vol 114 10717. Springer, Cham: Online. Retrieved from https://link.springer.com/chapter/10.1007%2F978-3-319-72598-7_2#citeas Mandiant. (2010). M-Trends: The advanced persistent threat. Retrieved from https://www2.fireeye.com/rs/fireye/images/PDF_MTrends_2010.pdf?mkt_tok=eyJpIjoi WW1RME9XRTVaV0prWkdVeiIsInQiOiJkakNWd1ViMjRXRXhHWUpkTkpcL054M 211bGp4SjhnT1wvdDJsYU1aa2Rna2d2eEw2ZFc1RWZjVGIyM2tZbm1NYzhjeWxyNS tibktYWmFQeWpPT1BNMkRwMlFEN1FkRjNPQ293NEl0Z2ZGMVM0ZDQ2Vm1iS VVGWDhuSmtZUTBNZHJRIn0%3D Margolis, G. (2013). The lack of HUMINT: A recurring intelligence problem. Global Security Studies 4(2), 43-60. Retrieved from https://www.academia.edu/37164453/The_Lack_of_HUMINT_A_Recurring_Intelligenc e_Problem Markoff, J., & Kramer, A. E. (2009, June 27). U.S. and Russia differ on a treaty for cyberspace. New York Times. Retrieved from https://www.nytimes.com/2009/06/28/world/28cyber.html Maronkova, B. (2018, April 10). NATO in the new hybrid warfare environment. Ukraine Analytica. Retrieved from http://ukraine-analytica.org/nato-in-the-new-hybrid-warfareenvironment/ Masters, J. (2018, October 23). NATO’s Trident Juncture exercises: What to know. Foreign Affiars. Council on Foreign Relations. Retrieved from https://www.cfr.org/article/natostrident-juncture-exercises-what-know 115 Mattis, J. N. (2018, January 19). Remarks by Secretary Mattis on the National Defense Strategy. Introduction Speech for National Defense Strategy, School of Advanced International Studies, Johns Hopkins University, Washington, DC. Retrieved from https://dod.defense.gov/News/Transcripts/TranscriptView/Article/1420042/remarks-by-secretary-mattis-on-the-national-defense-strategy/ Mattsson, P. (2015). Russian military thinking – A new generation of warfare. Journal on Baltic Security, 1(1), 61-70. Mazar, M. J. (2015). Mastering the gray zone: Understanding a changing era of conflict. Strategic Studies Institute and U.S. Army War College. Retrieved from https://ssi.armywarcollege.edu/pubs/download.cfm?q=1303 McCarthy, N. (2019, January 29). 45% of Russians are proud of Crimea’s annexation. Statista. Retrieved from https://www.statista.com/chart/16831/share-of-russians-who-are-mostproud-of-the-following/ McChrystal, S. (2011, February 21). It takes a network. Foreign Policy, 2(21) [online]. Retrieved from https://foreignpolicy.com/2011/02/21/it-takes-a-network/ McCulloh, T., & Johnson, R. (2013). Hybrid warfare. Tampa, FL: Joint Special Operations University. Retrieved from https://jsou.libguides.com/ld.php?content_id=2876897 McHale, J. (2015). Reducing Homicides using Social Network Analysis. Office of Justice Programs, Data‐Driven Justice Solutions. Retrieved from https://www.ojpdiagnosticcenter.org/blog/reducing‐homicides‐using‐social‐ network‐ analysis McKew, M. K. (2017, October). The Gerasimov Doctrine: It’s Russia’s new chaos theory of political warfare. And its probably being used on you. Politico, September/October, 116 2017. Retrieved from https://www.politico.com/magazine/story/2017/09/05/gerasimovdoctrine-russia-foreign-policy-215538 Mead, S. (2006). Unique file identification in the National Software Reference Library. Digital Investigation, 3, 138-150. Retrieved from https://www.researchgate.net/publication/222818580_Unique_file_identification_in_the_ National_Software_Reference_Library Medium. (2019, April 3). Digital Forensic Research Lab @ Atlantic Council. Retrieved from https://medium.com/dfrlab Meffert, C., Clark, D., Baggili, I., & Breitinger, F. (2017). Forensic state acquisition from Internet of Things (FSAIoT): A general framework and practical approach for IoT forensics through IoT device state acquisition. Presented at the 12th International Conference on Availability, Reliability and Security (ARES). Retrieved from https://doi.org/10.1016/j.future.2018.09.058 Meng, H., Thing, V. L. L., Cheng, Y., Dai, Z., & Zhang, L. (2018). A survey of Android exploits in the wild. Computers & Security, 76, 71-91. Retrieved from https://ac-els-cdncom.ezproxy.utica.edu/S0167404818301664/1-s2.0-S0167404818301664main.pdf?_tid=67745ec0-2070-421a-92461ef7fe2d62a3&acdnat=1552322002_68b680603e987c3c32cfce7d16442976 Michaels, J. (2018, January 31). How the U.S. is using terrorists’ smartphones and laptops to defeat them. USA Today. Retrieved from https://www.usatoday.com/story/news/world/2018/01/31/smartphones-computersterrorists-intelligence-agency-united-states/1079982001/ 117 Microsoft (2018, May 31). Service Control Manager. Retrieved from https://docs.microsoft.com/en-us/windows/desktop/services/service-control-manager Mizokami, K. (2017, July 27). Kaboom! Russian drone with thermite grenade blows up a billion dollars of Ukrainian ammo. Popular Mechanics. Retrieved from https://www.popularmechanics.com/military/weapons/news/a27511/russia-dronethermite-grenade-ukraine-ammo/ Moon, M. (2018). NATO special operations forces in the modern security environment [Draft report]. NATO Parliamentary Assembly, Defence and Security Committee. Retrieved from https://www.nato-pa.int/download-file?filename=sites/default/files/2018-04/2018 NATO SPECIAL OPERATIONS FORCES - DRAFT REPORT MOON - 064 DSCFC 18 E.pdf Morris, V. R. (2016, March 22) Identity and biometrics enabled intelligence (BEI) sharing for transnational threat actors. Small Wars Journal. Retrieved from https://stratcomcoe.org/victor-rmorris-identity-and-biometrics-enabled-intelligence-beisharing-transnational-threat-actors Naarttijärvi, M. (2016). Swedish police implementation of IMSI-catchers in a European law perspective. Computer Law & Security Review, 32(6), 852-867. Retrieved from https://ac-els-cdn-com.ezproxy.utica.edu/S0267364916301200/1-s2.0S0267364916301200-main.pdf?_tid=b98c4ce9-a4a2-4926-bcccc5df11e562c0&acdnat=1552334373_dc1b59388387fa625faafec537bdf782 National Institute of Standards and Technology. (2018). Framework for improving critical infrastructure cybersecurity. Version 1.1. Retrieved from https://doi.org/10.6028/NIST.CSWP.04162018 118 NATO. (2012). NATO logistics handbook. Retrieved from https://www.nato.int/docu/logien/logistics_hndbk_2012-en.pdf NATO. (2014, September). Wales Summit Declaration. Retrieved from https://www.nato.int/cps/ic/natohq/official_texts_112964.htm NATO. (2016, July). Warsaw Summit Communiqué. Retrieved from https://www.nato.int/cps/en/natohq/official_texts_133169.htm NATO. (2017, February). Warsaw summit key decisions. Retrieved from https://www.nato.int/nato_static_fl2014/assets/pdf/pdf_2017_02/20170206_1702factsheet-warsaw-summit-key-en.pdf NATO. (2018a, February). The NATO Command Structure. Retrieved from https://www.nato.int/nato_static_fl2014/assets/pdf/pdf_2018_02/1802-Factsheet-NATOCommand-Structure_en.pdf NATO. (2018b, February). NATO defence ministers take decisions to strength the Alliance. Retrieved from https://www.nato.int/cps/en/natohq/news_152125.htm NATO. (2018c, June 7). Three Allies establish special forces command. Retrieved from https://www.nato.int/cps/en/natohq/news_155347.htm NATO. (2018d, June 8). Third progress report on the implementation of the common set of proposals endorsed by EU and NATO Councils on 6 December 2016 and 5 December 2017. Retrieved from https://www.nato.int/nato_static_fl2014/assets/pdf/pdf_2018_06/20180608_180608-3rdJoint-progress-report-EU-NATO-eng.pdf 119 NATO. (2018e, July). Brussels summit declaration. Retrieved from https://www.nato.int/nato_static_fl2014/assets/pdf/pdf_2018_07/20180713_180711summit-declaration-eng.pdf NATO. (2019a, February 1). NATO-Russia: Setting the record straight. Retrieved from https://www.nato.int/cps/en/natohq/115204.htm#myths NATO. (2019b, February 13). Four Allies and one partner will create a regional Special Forces command. Retrieved from https://www.nato.int/cps/en/natohq/news_163360.htm NATO Collaborative Cyber Defense Centre of Excellence. (2019a, February). History. Retrieved from https://ccdcoe.org/about-us/ NATO Collaborative Cyber Defense Centre of Excellence. (2019b, March). Crossed Swords. Retrieved from https://ccdcoe.org/exercises/crossed-swords/ NATO Collaborative Cyber Defense Centre of Excellence. (2019c, March). Locked Shields. Retrieved from https://ccdcoe.org/exercises/locked-shields/ NATO Collaborative Cyber Defense Centre of Excellence. (2019d, March). Training. Retrieved from https://ccdcoe.org/training/?category=technical NATO Communications and Information Agency. (2019a). The NATO Communications and Information Agency: Leading NATO’s digital endeavour [Web page]. Retrieved from https://www.ncia.nato.int/About/Pages/About-the-NCI-Agency.aspx NATO Communications and Information Agency. (2019b). Cyber Security [Web page]. Retrieved from https://www.ncia.nato.int/Our-Work/Pages/Cyber-Security.aspx NATO Communications and Information Agency. (2019c). C4ISR & cyber training catalogue 2019. Retrieved from https://www.ncia.nato.int/SiteCollectionDocuments/ET%20Catalogue%202019.pdf 120 NATO Counter Improvised Explosive Device Centre of Excellence. (2019a). C-IED COE courses and events planned for 2019 and 2020. Retrieved from https://ciedcoe.org/index.php/docman/link-docs/299-course-catalogue-2019-2020/file NATO Counter Improvised Explosive Device Centre of Excellence. (2019b). Courses at C-IED COE. Retrieved from https://ciedcoe.org/index.php/courses-events/courses-at-c-ied-coe NATO Counter Improvised Explosive Device Centre of Excellence. (2019c). Status, Mission & Concept. Retrieved from https://ciedcoe.org/index.php/about-c-ied-coe/status-missionconcept NATO Military Police Centre of Excellence (2019, February 21). NATO MP COE Courses. Retrieved from https://www.mpcoe.org/activities/courses NATO Science and Technology Organization. (2014). NATO operational record: Collective analytical exploitation to inform operational analysis models and common operational planning factors. Retrieved from https://apps.dtic.mil/dtic/tr/fulltext/u2/a612386.pdf NATO Special Operations Headquarters. (2019a, March 24). NSOS course catalogue (20182019). Retrieved from https://www.nshq.nato.int/nsos/library/?TrainingPortalaction=public:document.download File&contentid=35D0E81A-5056-8B63-EFC70EAFD1683A87 NATO Special Operations Headquarters. (2019b, March 27). (NSOF-40) Digital media exploitation course. Retrieved from https://www.nshq.nato.int/nshq/training/?TrainingPortalaction=public:calendar.vieweven t&eventid=E0A9512B-5056-8396-DE3973F62C322523 NATO Special Operations Headquarters. (2019c, April 3). NSHQ vision, priority, focus areas, and mission. Retrieved from https://www.nshq.nato.int/nshq/ 121 NATO Stability Policing Centre of Excellence. (2016). Stability policing framework concept for forensics in NATO stabilization and reconstruction operations. Retrieved from http://www.nspcoe.org/docs/default-source/private/publications/stability-policingframework-concept-for-forensics-in-nato-stabilization-and-reconstructionoperations.pdf?sfvrsn=2 NATO Standardization Office. (2009). Allied joint doctrine for information operations (AJP3.10). Not retrieved due to classification. NATO Standardization Office. (2013a). NATO standardization agreement (4715) biometric data, interchange, watchlisting and reporting (Ed. 1). Retrieved from https://nso.nato.int/nso/zPublic/stanags/CURRENT/4715EFd01.pdf NATO Standardization Office. (2013b). Allied Joint Doctrine for Special Operations 3.5 (AJP3.5) (Ed. A, V1). Not retrieved due to classification. NATO Standardization Office. (2015). Allied Intelligence Publication 10 (AIntP-10), Technical Exploitation. Not retrieved due to classification. NATO Standardization Office. (2016). Countering threat anonymity: Biometrics in support of NATO operations and intelligence (AIntP-15). Not retrieved due to classification NATO Standardization Office. (2018a). NATO Glossary of terms and definitions (English and French). AAP-06. Retrieved from https://nso.nato.int/nso/nsdd/listpromulg.html NATO Standardization Office. (2018b). NATO military policy for information operations: Draft MC 0422/6. Working version as of September 11, 2018. Not retrieved due to classification. NATO Standardization Office. (2018c). Allied joint doctrine for countering improvised explosive devices (AJP-3.15). Edition C, Version 1. Not retrieved due to classification. 122 NATO Standardization Office. (2018d). Allied joint doctrine for logistics (AJP-4). Edition B, Version 1. Not retrieved due to classification. NATO Strategic Communications Centre of Excellence. (2016). Social media as a tool of hybrid warfare. Retrieved from https://www.stratcomcoe.org/download/file/fid/5314 Newman, L. H. (2019, January 3). The elite INTEL team still fighting Meltdown and Spectre. Wired. Retrieved from https://www.wired.com/story/intel-meltdown-spectre-storm/ Niculescu, B. R., & Coman, C. (2017). NATO Automated Biometric Identification System (NABIS). MTA Review, 27(2), 67-72. Retrieved from https://www.journal.mta.ro/articole/40/NATO%20Automated%20Biometric%20Identific ation%20System%20(NABIS).pdf Nilsen, T. (2019, March 4). Russian military officials arrive in Oslo as Norway puts GPS jamming facts on the table. The Barents Observer. Retrieved from https://thebarentsobserver.com/en/security/2019/03/russian-military-officials-arrive-oslonorway-provides-facts-gps-jamming Niva, S. (2013). Disappearing violence: JSOC and the Pentagon’s new cartography of networked warfare. Security Dialogue, 44(3), 185-202. Retrieved from https://journals-sagepubcom.ezproxy.utica.edu/doi/pdf/10.1177/0967010613485869 Noack, R. (2018, January 10). Everything we know so far about Russian election meddling in Europe. The Washington Post. Retrieved from https://www.washingtonpost.com/news/worldviews/wp/2018/01/10/everything-we-knowso-far-about-russian-election-meddling-in-europe/?utm_term=.7d0935cebe77 North Atlantic Treaty. (1949). Article 5. Retrieved from https://www.nato.int/cps/ie/natohq/official_texts_17120.htm 123 Novatel. (2019, March 28). GNSS frequencies and signals. Retrieved from https://www.novatel.com/support/known-solutions/gps-signals-and-related-frequencies/ Past, L. (2018). Cyberspace-just another domain of election interference? Strategic Analysis, August. Hybrid CoE. Retrieved from https://www.hybridcoe.fi/wpcontent/uploads/2018/10/Strategic-Analysis-2018-8-Past.pdf Patton, M., Gross, E., Chinn, R., Forbis, S., Walker, L., & Chen, H. (2014). Uninvited connections: A study of vulnerable devices on the Internet of Things (IoT). Presented at the 2014 IEEE Joint Intelligence and Security Informatics Conference. Tucson, AZ. Retrieved from https://ai.arizona.edu/sites/ai/files/AILabCybersecurityPapers/patton_et_al_2014_uninvit ed_connections_a_study_of_vulnerable_devices_on_the_internet_of_things.pdf Paxton, J. (2018, November 11). Trident Juncture and the information environment. NATO Review Magazine. Retrieved from https://www.nato.int/docu/review/2018/Also-in2018/trident-juncture-and-the-information-environment/EN/index.htm Payne III, R. L. (2017). Vehicle manipulation and forensics (Master’s thesis). Utica College. Retrieved from ttps://search-proquest-com.ezproxy.utica.edu/docview/1904411017?pqorigsite=gscholar Pearson, S. F. (2013). Creating the technology to biology bridge. Utica College. Retrieved from https://www.academia.edu/37764030/Creating_the_Technology_to_Biology_Bridge Pearson, S. F. (2018). Use of participative leadership during the chaos conflict component of a high-reliability organization (HRO) [Doctoral dissertation]. Retrieved from https://www.academia.edu/37780182/USE_OF_PARTICIPATIVE_LEADERSHIP_DU 124 RING_THE_CHAOS_COMPONENT_OF_A_HIGHRELIABILITY_ORGANIZATION_HRO_ Pearson, S. F., & Eggertsson, M. (2018). Proposing a change to the SIDSS framework for Technical Exploitation Operations: Advancing the inclusion of preparation & triage to the SIDSS Framework. Review of Business Research, 18(1), 49-58. Retrieved from https://www.academia.edu/37764030/Creating_the_Technology_to_Biology_Bridge Pell, S., & Soghoian, C. (2014). A lot more than a pen register, and less than a wiretap. Yale Journal of Law and Technology, 16(1), 134-171. Retrieved from http://digitalcommons.law.yale.edu/yjolt Perry, W. G. (2009). Information warfare: Assuring digital intelligence collection. Joint Special Operations University and the Strategic Studies Department. Retrieved from https://www.globalsecurity.org/military/library/report/2009/0907_jsou-paper-09-1.pdf Perry, B. (2016). Non-linear warfare in Ukraine: The critical role of information operations and special operations. Small Wars Journal. Retrieved from https://smallwarsjournal.com/jrnl/art/non-linear-warfare-in-ukraine-the-critical-role-ofinformation-operations-and-special-opera Pollitt, M., Casey, E., Jaquet-Chiffelle, D., & Gladyshev, P. (2019). A framework for harmonizing forensic science practices and digital/multimedia evidence. The Organization of Scientific Area Committees for Forensic Science (OSAC) Task Group on Digital/Multimedia Science, Technical Series 0002R1. Retrieved from https://www.nist.gov/sites/default/files/documents/2018/01/10/osac_ts_0002.pdf Polyakova, A., & Boyer, S. P. (2018). The future of political warfare: Russia, the West, and the coming age of global digital competition. The New Geopolitics-Europe. The Brookings 125 Institute. Retrieved from https://www.brookings.edu/wpcontent/uploads/2018/03/fp_20180316_future_political_warfare.pdf Portmess, L., & Romaya, B. (2015). Digital peacekeepers, drone surveillance and information fusion: A philosophical analysis of new peacekeeping. A Journal of Social and Political Theory 62(152), pp. 5-22. Retrieved from https://www-jstororg.ezproxy.utica.edu/stable/pdf/24719928.pdf Powner, D. (2009). National cybersecurity strategy: Key improvements are needed to strengthen the nation’s posture. Testimony before the Subcommittee on Emerging Threats, Cybersecurity, and Science and Technology, Committee on Homeland Security, House of Representatives. United States Government Accountability Office. Retrieved from https://obamawhitehouse.archives.gov/files/documents/cyber/Congress%20-%20GAOPowner-SFR_10Mar09.pdf Pozderka, Z. (2018). Commander’s critical information requirements. Applied Military Sciences, 1, p. 126-135. Retrieved from https://honvedelem.hu/files/files/112426/dr_2018_1_beliv_angol_szemle_126_135.pdf Pruitt, S. (2018, August 8). How a five-day war with Georgia allowed Russia to reassert its military might. History. Retrieved from https://www.history.com/news/russia-georgiawar-military-nato Radanliev, P., De Roure, D. C., Nicolescu, R., Huth, M., Montalvo, R. M., & Cannady, S., et al. (2018). Future developments in cyber risk assessment for the internet of things. Computers in Industry, 102, 12-22. Retrieved from https://ac-els-cdncom.ezproxy.utica.edu/S0166361518301817/1-s2.0-S0166361518301817- 126 main.pdf?_tid=38ec8d39-cf4f-42cc-ac50a1a2bc22035c&acdnat=1552334442_de0251bcbb95e82870cdf91d52fa9cbf Ramsay, G., & Robertshaw, S. (2018). Weaponising news RT, Sputnik and targeted disinformation. King’s College London. Retrieved from https://www.kcl.ac.uk/policyinstitute/assets/weaponising-news.pdf Raugh, D. L. (2016). Is the hybrid threat a true threat? Journal of Strategic Security, 9(2), 1-13. Retrieved from https://www-jstor-org.ezproxy.utica.edu/stable/pdf/26466774.pdf Reuters. (2018, March 20). Germany chooses Ulm for new proposed NATO logistics command. Retrieved from https://www.reuters.com/article/us-nato-germany/germany-chooses-ulmfor-new-proposed-nato-logistics-command-idUSKBN1GW1QM Rid, T., & Buchanan, B. (2015). Attributing cyber attacks. The Journal of Strategic Studies, 38(1-2), 4-37. Retrieved from http://dx.doi.org/10.1080/01402390.2014.977382 Roberts, J. Q. (2016). “Need authorities for the gray zone? Stop whining. Instead, help yourself to Title 100. Hell, take some Title 200 while your’re at it”. Prism, 6(3). Retrieved from https://www.jstor.org/stable/26470462?seq=2#metadata_info_tab_contents Robinson, L. (2013). The future of U.S. special operations forces. The Council on Foreign Relations, 66, April. Retrieved from https://www.cfr.org/content/publications/attachments/Special_Operations_CSR66.pdf Roblin, S. (2018, February 11). The Russian new warfare doctrine has the army worried enough to make a manual about it. The National Interest. Retrieved from https://nationalinterest.org/blog/the-buzz/the-russian-militarys-new-warfare-doctrine-hasthe-army-24439?page=0%2C1 127 Rotella, S. (2017a, November 10). Gangsters of the Mediterranean. The story of the Russian mob in Spain and the detectives who spent years trying to bring them down. The Atlantic, 11. Retrieved from https://www.theatlantic.com/international/archive/2017/11/russian-mobmallorca-spain/545504/ Rotella, S. (2017b, November 13). How Spain’s fight against gangsters revealed Russian power networks. Moneylife. Retreived from https://www.moneylife.in/article/how-spains-fightagainst-gangsters-revealed-russian-power-networks/52160.html Rubio Melón, F. J., Väisänen, T. U., & Pihelgas, M. (2018). EVE and ADAM: Situation awareness tools for NATO CCDCOE cyber exercises. NATO Science and Technology Organization. Retrieved from https://www.sto.nato.int/publications/STO%20Meeting%20Proceedings/STO-MP-SCI300/MP-SCI-300-10.pdf Sandhu, R., Sohal, A. S., & Sood, S. K. (2017). Identification of malicious edge devices in fog computing environments. Information Security Journal: A Global Perspective, 26(5), 213-228. San Pietro, D., Kammrath, B., & De Forest, P. (2018). Is forensic science in danger of extention? Science and Justice, 59, 199-202. Retrieved from https://www.sciencedirect.com/science/article/pii/S1355030618302454 Sanger, D. E. (2018). The perfect weapon: War, sabotage, and fear in the cyber age. New York: Crown. Schmitt, M. N. (Ed.). (2013). Tallinn manual on the international law applicable to cyber warfare. Cambridge: Cambridge University Press. 128 Schimtt, M. N. (Ed.). (2017). Tallinn Manual 2.0 (2nd ed.). Cambridge, UK: Cambridge University Press. Retrieved from www.cambridge.org/9781107177222. Schnaufer, T. (2017). Redefining hybrid warfare: Russia’s non-linear war against the west. Journal of Strategic Security, 10(1), 17-31. Retrieved from https://www-jstororg.ezproxy.utica.edu/stable/pdf/26466892.pdf Schouse, K. (2015). Actionability of cyber threat intelligence (Master’s thesis). Utica College. Retrieved from https://search.proquest.com/openview/a6758eb8711f52e5bb2d76182edf43dc/1?pqorigsite=gscholar&cbl=18750&diss=y Schroefl, J., & Kaufman, S. J. (2014). Hybrid actors, tactical variety: Rethinking asymmetric and hybrid war. Studies in Conflict & Terrorism, 37, 862-880. Retrieved from http://web.b.ebscohost.com.ezproxy.utica.edu/ehost/pdfviewer/pdfviewer?vid=3&sid=41 b97d06-396b-456d-9ca6-cf686d75456d%40pdc-v-sessmgr01 SERA. (2016). Hybrid warfare: Which European response. Session Europeenne des Responsables D’Armement (SERA) 28 reports. Retrieved from https://www.ihedn.fr/sites/default/files/atoms/files/rapport_sera28.pdf Shahbaz, A. (2018). Freedom on the net 2018: The rise of digital authoritarianism. Freedom House. Retrieved from https://freedomhouse.org/sites/default/files/FOTN_2018_Final%20Booklet_11_1_2018.p df Shea, J. (2017). How is NATO meeting the challenges of cyberspace? PRISM, 7(2), 18-29. Retrieved from https://www.jstor.org/stable/10.2307/26470515 129 Shuster, S. (2018, March 27). This KGB chief rang the alarm about Russia-U.S. cyberwars. No one listened. Time. Retrieved from http://time.com/5210728/russia-u-s-hackingcyberwar-kgb-soviet-union/ Sierra-Arévalo, M., & Papachristos, A. V. (2015). Social network analysis and gangs. In S. H. Decker, & D. C. Pyrooz (Eds.), The handbook of gangs. West Sussex, UK: John Wiley & Sons, Ltd. Retrieved from https://www.researchgate.net/publication/305355469_Social_Network_Analysis_and_Ga ngs Simkin, H. R. (2018). The future ODA 2035-2050. Presented as part of the TRADOC G2’s “Soldier 2050” Call for Ideas at the SRI International Mad Scientist Conference, March 8-9. Retrieved from https://smallwarsjournal.com/jrnl/art/future-oda-2035-2050 Sorrel, C. (2018, January 10). Your iPhone tracks every place you visit. Here’s how to see the map. Cult of Mac. Retreived from https://www.cultofmac.com/522515/how-to-seeiphone-significant-locations-map/ Symantec. (2019). Internet security threat report 2019. Volume 24, February. Retrieved from http://linkinghub.elsevier.com/retrieve/pii/S1353485805001947 Shamir, E., & Ben-Ari, E. (2016). The rise of special operations: Generalized specialization, boundary spanning, and military autonomy. Journal of Strategic Studies 41(3), 1–37. Shultz, R. (2016). Military innovation in war: It takes a learning organization a case study of Task Force 714 in Iraq. Tampa, FL: Joint Special Operations University Smith, B., Thomas, M. & Tranchemontagne, M. (2014). Understanding the enemy: The enduring value of technical and forensic exploitation. Joint Force Quarterly, 75 (4th Quarter), 123128. Retrieved from https://apps.dtic.mil/dtic/tr/fulltext/u2/a622235.pdf 130 Spidalieri, F., & McArdle, J. (2016). Transforming the next generation of military leaders into cyber-strategic leaders: The role of cybersecurity education in US service academies. The Cyber Defense Review, 1, p. 141-164. Retrieved from https://www-jstororg.ezproxy.utica.edu/stable/pdf/26267304.pdf Sputnik. (2018, June 1). NATO members agree to locate new logistics command in Germany’s Ulm. Retrieved from https://sputniknews.com/europe/201806011064999968-natogermany-logistics/ Steder, F. B. (2016). Introduction: The theory, history, and current state of hybrid warfare. In Skinner E. (Eds.) Countering Hybrid Warfare: The best uses of SOF in a pre-Article V scenario. CTX, 6(4), 7-18. Retrieved from https://www.ffi.no/no/Publikasjoner/Documents/CTX_Countering Hybrid Warfare. The best use of SOF in a pre-article V Scenario.pdf Stolk, K. (2016, May 9). Testing direction finder at GPS jamming trials. NATO Communications and Information Agency. Retrieved from https://www.ncia.nato.int/NewsRoom/Pages/160905-Sennybridge-GPS-JammingTrials.aspx Richardson, M. (2016). Afterward. In Skinner E. (Eds.) Countering Hybrid Warfare: The best uses of SOF in a pre-Article V scenario. CTX, 6(4), 7-18. Retrieved from https://www.ffi.no/no/Publikasjoner/Documents/CTX_Countering Hybrid Warfare. The best use of SOF in a pre-article V Scenario.pdf TASS Russian News Agency. (2018, December 18). Reconnaissance and attack drones to arrive for Russian Army from 2019. Retrieved from http://tass.com/defense/1036662 131 Taylor, C. (2017). It’s time for cavalry to get serious about cyber reconnaissance. eARMOUR, Winter 2017. Retrieved from https://www.benning.army.mil/Armor/eARMOR/content/issues/2018/Fall/4Taylor18.pdf Tetzschner, M. (2018). Nordic Council: Cyber threats should be tackled by way of closer cooperation. Nordic Co-operation. Retrieved from https://www.norden.org/en/news/nordiccouncil-cyber-threats-should-be-tackled-way-closer-co-operation The Baltic Course. (2017, August) Lithuanian DefMin: Software linked with Russia has spying gaps. Retrieved from http://www.baltic-course.com/eng/legislation/?doc=132172 The Economy Journal. (2019, March 31). More than 3000 companies have left Catalonia after the referendum. Retrieved from https://www.theeconomyjournal.eu/textodiario/mostrar/973022/more-than-3000-companies-have-left-catalonia-after-thereferendum Thornton, R. (2018, January 17). Current Russian and Chinese way of warfare: The end (?) of military violence in peer-state conflict. Defence-in-Depth, King’s College London. Retrieved from https://defenceindepth.co/2018/01/17/current-russian-and-chinese-waysof-warfare-the-end-of-military-violence-in-peer-state-conflict/ Tikk, E., Kaska, K., Rünnimeri, K., Kert, M., Talihärm, & Vihul, L. (2008). Cyber attacks against Georgia: Legal lessons identified. The CCDCOE Legal Task Team. Retrieved from http://www.ismlab.usf.edu/isec/files/Georgia-Cyber-Attack-NATO-Aug-2008.pdf Tikk, E., Kaska, K., & Vihul., L. (2010). International cyber incidents: Legal considerations. Tallinn: CCDCOE Publications. Transparency International. (2018). Corruption perceptions index 2018. Retrieved from https://www.transparency.org/cpi2018 132 Treverton, G. F. (2018). The intelligence challenges of hybrid threats: Focus on cyber and virtual realm. Swedish Defence University. Retrieved from http://fhs.divaportal.org/smash/get/diva2:1250560/FULLTEXT01.pdf Trevithick, J. (2017, September 26). New US Army manual shows it’s worried about Russia’s hybrid warfare tactics. The Drive. Retrieved from http://thedrive.com/the-warzone/14647/new-us-army-manual-shows-its-worried-about-russias-hybrid-warfare-tactics Union of Concerned Scientists Satellite Database. (2019, February 21). The countries with the most satellites in space. Retrieved from Bucholtz, K., Statista, https://www.statista.com/chart/17107/countries-with-the-most-satellites-in-space/ United Nations. (2011). Criminal intelligence manual for analysts. United Nations Office on Drugs and Crime. Retrieved from https://www.unodc.org/documents/organizedcrime/Law-Enforcement/Criminal_Intelligence_for_Analysts.pdf Urquhart, L., & McAuley, D. (2018). Avoiding the internet of insecure industrial things. Computer Law and Security Review, 34(3), 450-466. Retrieved from https://ac-els-cdncom.ezproxy.utica.edu/S0167404818301664/1-s2.0-S0167404818301664main.pdf?_tid=67745ec0-2070-421a-92461ef7fe2d62a3&acdnat=1552322002_68b680603e987c3c32cfce7d16442976 United States Army Special Operations Command. (2019a, March 23). Operator Advanced Course (OAC). Retrieved from https://www.soc.mil/swcs/SWCS%20Courses/COURSE%20PDF/6th%20Bn/SOF%20Se nsitive%20Site%20Exploitation,%20Operator%20Advance%20Course%20SSE%202EF258011-F98.pdf 133 United States Army Special Operations Command. (2019b, March 23). Exploitation Analysis Course (EAC) welcome letter. Retrieved from https://www.soc.mil/swcs/SWCS%20Courses/COURSE%20PDF/6th%20Bn/EXPLOIT ATION%20ANALYSIS%20CENTER%20COURSE%20Welcome%20letter.docx United States Cyber Command. (2019, February). U.S. Cyber Command history. Retrieved from https://www.cybercom.mil/About/History/ United States Defense Forensics and Biometrics Agency. (2019). Purpose: Enhancing readiness through identity activities. Retrieved from https://www.dfba.mil/about/about-dfba.html United States Department of the Army. (2015). Site exploitation. Army Techniques Publication ATP 3-90.15. Retrieved from https://fas.org/irp/doddir/army/atp3-90-15.pdf United States Department of Justice. (2017, November 30). Russian cyber-criminal sentenced to 14 years in prison for role in organized cybercrime ring responsible for $50 million in online identity theft and $9 million bank fraud conspiracy. Department of Justice Office of Public Affairs. Retrieved from https://www.justice.gov/opa/pr/russian-cyber-criminalsentenced-14-years-prison-role-organized-cybercrime-ring-responsible United States Department of Transportation. (2018). Freight facts and figures 2017. Retrieved from https://www.bts.gov/sites/bts.dot.gov/files/docs/FFF_2017_Full_June2018revision.pdf United States Department of Defense. (2011). DoD Forensic Enterprise (DFE). DoD Directive 5205.15E, April 26, 2011. Retrieved from https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodd/520515e.pdf United States Department of Defense. (2013). Joint Publication 2-0 on Joint Intelligence. Retrieved from https://fas.org/irp/doddir/dod/jp2_0.pdf 134 United States Department of Defense. (2014). Joint Publication on Special Operations JP 3-05. Retrieved from https://fas.org/irp/doddir/dod/jp3_05.pdf United States Department of Defense. (2016a). Contracts for May 25, 2016. Retrieved from https://dod.defense.gov/News/Contracts/Contract-View/Article/781719/ United States Department of Defense. (2016b). Joint Doctrine Note 2-16 on Identity Activities (2016). Retrieved from https://fas.org/irp/doddir/dod/jdn2_16.pdf United States Government Accountability Office. (2017a). Internet of Things. Enhanced assessments and guidance are needed to address security risks. GAO Report to Congressional Committees, GAO-17-668. Retrieved from https:// www.gao.gov/ assets/690/686203.pdf. United States Joint Special Operations University. (2018). Special operations research Topics 2018: Revised edition for academic year 2019. JSOU Press. Retrieved from https://jsou.libguides.com/ld.php?content_id=41898487 United States National Cybersecurity and Communications Integration Center. (2018, March 15). Alert (TA18-074A) Russian government cyber activity targeting energy and other critical infrastructure sectors. Retrieved from https://www.us-cert.gov/ncas/alerts/TA18074A Valeriano, B., & Jensen, B. (2019). The myth of the cyber offensive: The case for restraint. Policy Analysis, 862, 66-74. Retrieved from https://www.cato.org/publications/policyanalysis/myth-cyber-offense-case-restraint#full Van der Walt, C., & Pillarisetty, S. (2018). Don’t eat spaghetti with a spoon. Presented at Black Hat Europe, 2018. Retrieved from https://i.blackhat.com/eu-18/Thu-Dec-6/eu-18-VdwaltDont-Eat-Spaghetti-With-A-Spoon-2.pdf 135 Van Puyvelde, D. (2015). Hybrid war – does it even exist? NATO Review Magazine. Retrieved from https://www.nato.int/DOCU/review/2015/Also-in-2015/hybrid-modern-futurewarfare-russia-ukraine/EN/index.htm Vann, P. (2017, September 18). Casting a light on BlackEnergy. ThreatConnect. Retrieved from https://threatconnect.com/casting-a-light-on-blackenergy/ Vesti Nedeli. (2019). After Putin’s warning, Russian TV lists nuclear targets in U.S. The Moscow Times. Retrieved from https://www.themoscowtimes.com/2019/02/25/afterputins-warning-russian-tv-lists-nuclear-targets-in-us-a64614 Verner, D., Grigas, A., & Petit, F. (2019). Assessing energy dependency in the age of hybrid threats. Hybrid CoE. Retrieved from https://www.hybridcoe.fi/wpcontent/uploads/2019/02/Assessing_Energy_Dependency_in_the_Age_of_Hybrid_Threat s-HybridCoE.pdf Vores, C. S. (2012). Taking the lead in professional growth: The development of a NATO SOF intelligence officer (Master’s thesis). U.S. Naval Postgraduate School. Monterey, CA. Retrieved from https://apps.dtic.mil/dtic/tr/fulltext/u2/a574674.pdf Weber, G. (2018, November 6). Data rich and information poor (DRIP): The adversary of lethality. The Strategy Bridge. Retrieved from https://thestrategybridge.org/thebridge/2018/11/6/data-rich-and-information-poor-dripthe-adversary-of-lethality Yun, S., Faraj, S., & Sims, H. P., Jr. (2005). Contingent leadership and effectiveness of trauma resuscitation teams. The Journal of Applied Psychology, 90(6), 1288– 1296. doi:10.1037/0021-9010.90.6.1288 136 Appendix A: Forensic Questions to Reduce Assumptive Associations Below are sample questions taken from Pollitt et al. (2019). Authentication: • Files are identical at the binary level. • Files are identical at the semantic level. • Files are created at the time indicated by the file system. • Files are unaltered. • The source device’s clock was accurate. • Files were geolocated to X location. • The source device is infected/compromised. • The communication’s source is authoritative. Identification • A device was last accessed by individual X. • A process was executed by process/individual X. • This file was last accessed by individual X. • Device X was used to capture this media. Classification • Retrieved data is X type, e.g. credit card number, IP address, password. • This segment of a malware payload is ransomware. Reconstruction • Reconstruct communication between individuals/entities. • Reconstruct traces of data deletion, including time and date. • Reconstruct a timeline of user activity on a device. 137 • Reconstruct connection between this device and cloud-based storage. • Reconstruct access control restrictions on a file share. Evaluation • List and evaluate all possible explanations of activity. 138