Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3474718.3475718acmotherconferencesArticle/Chapter ViewAbstractPublication PagescsetConference Proceedingsconference-collections
research-article

D2U: Data Driven User Emulation for the Enhancement of Cyber Testing, Training, and Data Set Generation

Published: 07 September 2021 Publication History

Abstract

Whether testing intrusion detection systems, conducting training exercises, or creating data sets to be used by the broader cybersecurity community, realistic user behavior is a critical component of a cyber range. Existing methods either rely on network level data or replay recorded user actions to approximate real users in a network. Our work produces generative models trained on actual user data (sequences of application usage) collected from endpoints. Once trained to the user’s behavioral data, these models can generate novel sequences of actions from the same distribution as the training data. These sequences of actions are then fed to our custom software via configuration files, which replicate those behaviors on end devices. Notably, our models are platform agnostic and could generate behavior data for any emulation software package. In this paper we present our model generation process, software architecture, and an investigation of the fidelity of our models. Specifically, we consider two different representations of the behavioral sequences, on which three standard generative models for sequential data—Markov Chain, Hidden Markov Model, and Random Surfer—are employed. Additionally, we examine adding a latent variable to faithfully capture time-of-day trends. Best results are observed when sampling a unique next behavior (regardless of the specific sequential model used) and the duration to take the behavior, paired with the temporal latent variable. Our software is currently deployed in a cyber range to help evaluate the efficacy of defensive cyber technologies, and we suggest additional ways that the cyber community as a whole can benefit from more realistic user behavior emulation.

References

[1]
William H Allen. 2007. Mixing wheat with the chaff: Creating useful test data for ids evaluation. IEEE Security & Privacy 5, 4 (2007), 65–67.
[2]
A Botello, J Lin, D Mozzacco, JE Sutton, M Spraragen, J Blythe, and M Zyda. 2010. An Agent Architecture for Large-scale Security Simulation. (2010).
[3]
Timothy M Braje. 2016. Advanced tools for cyber ranges. Technical Report. MIT Lincoln Laboratory Lexington United States.
[4]
Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems 30, 1-7 (1998), 107–117.
[5]
Preetam Dutta, Gabriel Ryan, Aleksander Zieba, and Salvatore Stolfo. 2018. Simulated user bots: Real time testing of insider threat detection systems. In 2018 IEEE Security and Privacy Workshops (SPW). IEEE, 228–236.
[6]
Alexis Gabadinho, Gilbert Ritschard, Nicolas S. Müller, and Matthias Studer. 2011. Analyzing and Visualizing State Sequences in R with TraMineR. Journal of Statistical Software 40, 4 (2011). https://doi.org/10.18637/jss.v040.i04
[7]
Paul A Gagniuc. 2017. Markov chains: from theory to implementation and experimentation. John Wiley & Sons.
[8]
Wael H Gomaa, Aly A Fahmy, 2013. A survey of text similarity approaches. International Journal of Computer Applications 68, 13(2013), 13–18.
[9]
Richard W Hamming. 1950. Error detecting and error correcting codes. The Bell system technical journal 29, 2 (1950), 147–160.
[10]
William R Knight. 1966. A computer method for calculating Kendall’s tau with ungrouped data. J. Amer. Statist. Assoc. 61, 314 (1966), 436–439.
[11]
Vijay Kothari, Jim Blythe, Sean W Smith, and Ross Koppel. 2015. Measuring the security impacts of password policies using cognitive behavioral agent-based modeling. In Proceedings of the 2015 Symposium and Bootcamp on the Science of Security. 1–9.
[12]
Samir Mammadov, Dhanish Mehta, Evan Stoner, and Marco M Carvalho. 2017. High fidelity adaptive cyber emulation. In 2017 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 1–8.
[13]
Frederic Massicotte, Francois Gagnon, Yvan Labiche, Lionel Briand, and Mathieu Couture. 2006. Automatic evaluation of intrusion detection systems. In 2006 22nd Annual Computer Security Applications Conference (ACSAC’06). IEEE, 361–370.
[14]
John McHugh. 2000. Testing intrusion detection systems: a critique of the 1998 and 1999 darpa intrusion detection system evaluations as performed by lincoln laboratory. ACM Transactions on Information and System Security (TISSEC) 3, 4(2000), 262–294.
[15]
Péter Megyesi, Géza Szabó, and Sándor Molnár. 2015. User behavior based traffic emulator: A framework for generating test data for DPI tools. Computer Networks 92(2015), 41–54.
[16]
Lawrence R Rabiner. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 2 (1989), 257–286.
[17]
Lee M Rossey, Robert K Cunningham, David J Fried, Jesse C Rabek, Richard P Lippmann, Joshua W Haines, and Marc A Zissman. 2002. LARIAT: Lincoln adaptable real-time information assurance testbed. In Proceedings, IEEE Aerospace Conference, Vol. 6. IEEE, 6–6.
[18]
Iman Sharafaldin, Arash Habibi Lashkari, and Ali A Ghorbani. 2018. Toward generating a new intrusion detection dataset and intrusion traffic characterization. In ICISSP. 108–116.
[19]
Matthias Studer and Gilbert Ritschard. 2014. A comparative review of sequence dissimilarity measures. (2014). https://doi.org/10.12682/lives.2296-1658.2014.33
[20]
Dustin D Updyke, Geoffrey B Dobson, Thomas G Podnar, Luke J Osterritter, Benjamin L Earl, and Adam D Cerini. 2018. Ghosts in the Machine: A Framework for Cyber-Warfare Exercise NPC Simulation. Technical Report. CARNEGIE-MELLON UNIV PITTSBURGH PA PITTSBURGH United States.
[21]
John Wroclawski, Terry Benzel, Jim Blythe, Ted Faber, Alefiya Hussain, Jelena Mirkovic, and Stephen Schwab. 2016. DETERLab and the DETER Project. In The GENI Book. Springer, 35–62.
[22]
Stefano Zanero. 2007. Flaws and frauds in the evaluation of IDS/IPS technologies. In Proc. of FIRST. Citeseer.

Cited By

View all
  • (2023)Enhancing Honeypot Fidelity with Real-Time User Behavior Emulation2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S)10.1109/DSN-S58398.2023.00041(146-150)Online publication date: Jun-2023

Index Terms

  1. D2U: Data Driven User Emulation for the Enhancement of Cyber Testing, Training, and Data Set Generation
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Other conferences
        CSET '21: Proceedings of the 14th Cyber Security Experimentation and Test Workshop
        August 2021
        95 pages
        ISBN:9781450390651
        DOI:10.1145/3474718
        Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 07 September 2021

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. data driven
        2. data sets
        3. experimental infrastructure
        4. user emulation

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Conference

        CSET '21

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)47
        • Downloads (Last 6 weeks)5
        Reflects downloads up to 18 Aug 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2023)Enhancing Honeypot Fidelity with Real-Time User Behavior Emulation2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S)10.1109/DSN-S58398.2023.00041(146-150)Online publication date: Jun-2023

        View Options

        Get Access

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media