Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/2694344.2694386acmconferencesArticle/Chapter ViewAbstractPublication PagesasplosConference Proceedingsconference-collections
research-article

Nested Kernel: An Operating System Architecture for Intra-Kernel Privilege Separation

Published: 14 March 2015 Publication History
  • Get Citation Alerts
  • Abstract

    Monolithic operating system designs undermine the security of computing systems by allowing single exploits anywhere in the kernel to enjoy full supervisor privilege. The nested kernel operating system architecture addresses this problem by "nesting" a small isolated kernel within a traditional monolithic kernel. The "nested kernel" interposes on all updates to virtual memory translations to assert protections on physical memory, thus significantly reducing the trusted computing base for memory access control enforcement. We incorporated the nested kernel architecture into FreeBSD on x86-64 hardware while allowing the entire operating system, including untrusted components, to operate at the highest hardware privilege level by write-protecting MMU translations and de-privileging the untrusted part of the kernel. Our implementation inherently enforces kernel code integrity while still allowing dynamically loaded kernel modules, thus defending against code injection attacks. We also demonstrate that the nested kernel architecture allows kernel developers to isolate memory in ways not possible in monolithic kernels by introducing write-mediation and write-logging services to protect critical system data structures. Performance of the nested kernel prototype shows modest overheads: <1% average for Apache and 2.7% for kernel compile. Overall, our results and experience show that the nested kernel design can be retrofitted to existing monolithic kernels, providing important security benefits.

    References

    [1]
    AMD64 architecture programmers manual volume 2: System programming. Manual, Advancd Micro Devices, 2006.
    [2]
    Intel 64 and IA-32 architectures software developers manual. Manual 325384-051US, Intel, June 2014.
    [3]
    M. Accetta, R. Baron, W. Bolosky, D. Golub, R. Rashid, A. Tevanian, and M. Young. Mach: A new kernel foundation for UNIX development. In Proceedings of the USENIX Annual Technical Conference, USENIX ATC'10, pages 93--112, Altanta, GA, USA, 1986. USENIX Association.
    [4]
    M. Aiken, M. Fhndrich, C. Hawblitzel, G. Hunt, and J. Larus. Deconstructing process isolation. In Proceedings of the 2006 Workshop on Memory System Performance and Correctness, MSPC '06, pages 1--10, New York, NY, USA, 2006. ACM.
    [5]
    argp and Karl. Exploiting UMA, FreeBSD's kernel memory allocator.:: Phrack Magazine ::., 0x0d(0x42), Nov. 2009.
    [6]
    S. Bahram, X. Jiang, Z. Wang, M. Grace, J. Li, D. Srinivasan, J. Rhee, and D. Xu. DKSM: Subverting virtual machine introspection for fun and profit. In Proceedings of the 2010 29th IEEE Symposium on Reliable Distributed Systems, SRDS '10, pages 82--91, Washington, DC, USA, 2010. IEEE Computer Society.
    [7]
    P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, SOSP '03, pages 164--177, New York, NY, USA, 2003. ACM.
    [8]
    A. Belay, A. Bittau, A. Mashtizadeh, D. Terei, D. Mazires, and C. Kozyrakis. Dune: Safe user-level access to privileged CPU features. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI'12, pages 335--348, Berkeley, CA, USA, 2012. USENIX Association.
    [9]
    B. N. Bershad, S. Savage, P. Pardyak, E. G. Sirer, M. E. Fiuczynski, D. Becker, C. Chambers, and S. Eggers. Extensibility safety and performance in the SPIN operating system. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles, SOSP '95, pages 267--283, New York, NY, USA, 1995. ACM.
    [10]
    M. Castro, M. Costa, J.-P. Martin, M. Peinado, P. Akritidis, A. Donnelly, P. Barham, and R. Black. Fast byte-granularity software fault isolation. In Proceedings of the ACM SIGOPS 22nd symposium on Operating Systems Principles, SOSP '09, pages 45--58, New York, NY, USA, 2009. ACM.
    [11]
    S. Chen, J. Xu, E. C. Sezer, P. Gauriar, and R. K. Iyer. Non-control-data attacks are realistic threats. In Proceedings of the 14th Conference on USENIX Security Symposium - Volume 14, SSYM'05, pages 12--12, Berkeley, CA, USA, 2005. USENIX Association.
    [12]
    A. Chou, J. Yang, B. Chelf, S. Hallem, and D. Engler. An empirical study of operating systems errors. In Proceedings of the Eighteenth ACM Symposium on Operating Systems Principles, SOSP '01, pages 73--88, New York, NY, USA, 2001. ACM.
    [13]
    J. Criswell, N. Dautenhahn, and V. Adve. KCoFI: Complete control-flow integrity for commodity operating system kernels. In Proceedings of the 2014 IEEE Symposium on Security and Privacy, SP '14, pages 292--307, Washington, DC, USA, 2014. IEEE Computer Society.
    [14]
    J. Criswell, N. Dautenhahn, and V. Adve. Virtual ghost: Protecting applications from hostile operating systems. In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS '14, pages 81--96, New York, NY, USA, 2014. ACM.
    [15]
    J. Criswell, N. Geoffray, and V. Adve. Memory safety for low-level software/hardware interactions. In Proceedings of the 18th Conference on USENIX Security Symposium, SSYM'09, pages 83--100, Berkeley, CA, USA, 2009. USENIX Association.
    [16]
    J. Criswell, A. Lenharth, D. Dhurjati, and V. Adve. Secure virtual architecture: A safe execution environment for commodity operating systems. In Proceedings of Twenty-first ACM SIGOPS Symposium on Operating Systems Principles, SOSP '07, pages 351--366, New York, NY, USA, 2007. ACM.
    [17]
    D. R. Engler, M. F. Kaashoek, and J. O'Toole, Jr. Exokernel: An operating system architecture for application-level resource management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles, SOSP '95, pages 251--266, New York, NY, USA, 1995. ACM.
    [18]
    U. Erlingsson, M. Abadi, M. Vrable, M. Budiu, and G. C. Necula. XFI: Software guards for system address spaces. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation, OSDI '06, pages 75--88, Berkeley, CA, USA, 2006. USENIX Association.
    [19]
    T. Garfinkel and M. Rosenblum. A virtual machine introspection based architecture for intrusion detection. In Proceedings of the Network and Distributed System Security Symposium, NDSS 2003, San Diego, California, USA. The Internet Society, 2003.
    [20]
    O. S. Hofmann, S. Kim, A. M. Dunn, M. Z. Lee, and E. Witchel. InkTag: Secure applications on an untrusted operating system. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS '13, pages 265--278, New York, NY, USA, 2013. ACM.
    [21]
    S. A. Hofmeyr, S. Forrest, and A. Somayaji. Intrusion detection using sequences of system calls. J. Comput. Secur., 6(3):151--180, Aug. 1998.
    [22]
    N. Honarmand, N. Dautenhahn, J. Torrellas, S. T. King, G. Pokam, and C. Pereira. Cyrus: Unintrusive application-level record-replay for replay parallelism. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS '13, pages 193--206, New York, NY, USA, 2013. ACM.
    [23]
    N. Honarmand and J. Torrellas. Replay debugging: Leveraging record and replay for program debugging. In Proceeding of the 41st Annual International Symposium on Computer Architecuture, ISCA '14, pages 445--456, Piscataway, NJ, USA, 2014. IEEE Press.
    [24]
    B. Jain, M. B. Baig, D. Zhang, D. E. Porter, and R. Sion. SoK: Introspections on trust and the semantic gap. In Proceedings of the 2014 IEEE Symposium on Security and Privacy, SP '14, pages 605--620, Washington, DC, USA, 2014. IEEE Computer Society.
    [25]
    V. P. Kemerlis, M. Polychronakis, and A. D. Keromytis. ret2dir: Rethinking kernel isolation. In Proceedings of the 23rd USENIX Conference on Security Symposium, SEC'14, pages 957--972, Berkeley, CA, USA, 2014. USENIX Association.
    [26]
    S. T. King and P. M. Chen. Backtracking intrusions. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, SOSP '03, pages 223--236, New York, NY, USA, 2003. ACM.
    [27]
    J. Kong. Designing BSD Rootkits. No Starch Press, San Francisco, CA, USA, 2007.
    [28]
    J. Liedtke. On micro-kernel construction. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles, SOSP '95, pages 237--250, New York, NY, USA, 1995. ACM.
    [29]
    Y. Mao, H. Chen, D. Zhou, X. Wang, N. Zeldovich, and M. F. Kaashoek. Software fault isolation with API integrity and multi-principal modules. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP '11, pages 115--128, New York, NY, USA, 2011. ACM.
    [30]
    L. McVoy and C. Staelin. lmbench: Portable tools for performance analysis. In Proceedings of the 1996 Annual Conference on USENIX Annual Technical Conference, ATEC '96, pages 23--23, Berkeley, CA, USA, 1996. USENIX Association.
    [31]
    Microsoft. Kernel patch protection: frequently asked questions (windows drivers), 2007.
    [32]
    P. Montesinos, M. Hicks, S. T. King, and J. Torrellas. Capo: A software-hardware interface for practical deterministic multiprocessor replay. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIV, pages 73-- 84, New York, NY, USA, 2009. ACM.
    [33]
    D. Mutz, F. Valeur, G. Vigna, and C. Kruegel. Anomalous system call detection. ACM Trans. Inf. Syst. Secur., 9(1):61--93, Feb. 2006.
    [34]
    E. I. Organick. The Multics System: An Examination of Its Structure. MIT Press, Cambridge, MA, USA, 1972.
    [35]
    B. D. Payne, M. Carbone, M. Sharif, and W. Lee. Lares: An architecture for secure active monitoring using virtualization. In Proceedings of the 2008 IEEE Symposium on Security and Privacy, SP '08, pages 233--247, Washington, DC, USA, 2008. IEEE Computer Society.
    [36]
    G. Pokam, K. Danne, C. Pereira, R. Kassa, T. Kranich, S. Hu, J. Gottschlich, N. Honarmand, N. Dautenhahn, S. T. King, and J. Torrellas. QuickRec: Prototyping an intel architecture extension for record and replay of multithreaded programs. In Proceedings of the 40th Annual International Symposium on Computer Architecture, ISCA '13, pages 643--654, New York, NY, USA, 2013. ACM.
    [37]
    C. Ries. Defeating windows personal firewalls: Filtering methodologies, attacks, and defenses. Technical report, 2005.
    [38]
    R. Riley, X. Jiang, and D. Xu. Guest-transparent prevention of kernel rootkits with VMM-based memory shadowing. In Proceedings of the 11th International Symposium on Recent Advances in Intrusion Detection, RAID '08, pages 1--20, Berlin, Heidelberg, 2008. Springer-Verlag.
    [39]
    J. M. Rushby. Design and verification of secure systems. In Proceedings of the Eighth ACM Symposium on Operating Systems Principles, SOSP '81, pages 12--21, New York, NY, USA, 1981. ACM.
    [40]
    J. H. Saltzer. Protection and the control of information sharing in multics. Commun. ACM, 17(7):388--402, July 1974.
    [41]
    J. H. Saltzer and M. D. Schroeder. The protection of information in computer systems. Proceedings of the IEEE, 63(9):1278--1308, 1975.
    [42]
    T. Saulpaugh and C. A. Mirho. Inside the JavaOS operating system. Addison-Wesley Reading, 1999.
    [43]
    A. Seshadri, M. Luk, N. Qu, and A. Perrig. SecVisor: A tiny hypervisor to provide lifetime kernel code integrity for commodity OSes. In Proceedings of Twenty-first ACM SIGOPS Symposium on Operating Systems Principles, SOSP '07, pages 335--350, New York, NY, USA, 2007. ACM.
    [44]
    M. I. Sharif, W. Lee, W. Cui, and A. Lanzi. Secure in-VM monitoring using hardware virtualization. In Proceedings of the 16th ACM Conference on Computer and Communications Security, CCS '09, pages 477--487, New York, NY, USA, 2009. ACM.
    [45]
    G. E. Suh, D. Clarke, B. Gassend, M. van Dijk, and S. Devadas. AEGIS: Architecture for tamper-evident and tamper-resistant processing. In Proceedings of the 17th Annual International Conference on Supercomputing, ICS '03, pages 160--171, New York, NY, USA, 2003. ACM.
    [46]
    M. M. Swift, B. N. Bershad, and H. M. Levy. Improving the reliability of commodity operating systems. ACM Trans. Comput. Syst., 23(1):77--110, Feb. 2005.
    [47]
    A. Tereshkin. Rootkits: Attacking personal firewalls. In Proceedings of the Black Hat USA 2006 Conference, 2006.
    [48]
    I. Unified EFI. Unified extensible firmware interface specification: Version 2.2d, November 2010.
    [49]
    D. Wagner and P. Soto. Mimicry attacks on host-based intrusion detection systems. In Proceedings of the 9th ACM Conference on Computer and Communications Security, CCS '02, pages 255--264, New York, NY, USA, 2002. ACM.
    [50]
    Z. Wang and X. Jiang. HyperSafe: A lightweight approach to provide lifetime hypervisor control-flow integrity. In Proceedings of the 2010 IEEE Symposium on Security and Privacy, SP '10, pages 380--395, Washington, DC, USA, 2010. IEEE Computer Society.
    [51]
    Z. Wang, X. Jiang, W. Cui, and P. Ning. Countering kernel rootkits with lightweight hook protection. In Proceedings of the 16th ACM Conference on Computer and Communications Security, CCS '09, pages 545--554, New York, NY, USA, 2009. ACM.
    [52]
    C. Warrender, S. Forrest, and B. A. Pearlmutter. Detecting intrusions using system calls: Alternative data models. In 1999 IEEE Symposium on Security and Privacy, SP '99, pages 133--145, Oakland, California, USA, May 1999. IEEE Computer Society.
    [53]
    D. Wheeler. SLOCCount, 2015. http://www.dwheeler.com/sloccount/.
    [54]
    C. Wright. Para-virtualization interfaces, 2006. http://lwn.net/Articles/194340.
    [55]
    X. Xiong and P. Liu. SILVER: Fine-grained and transparent protection domain primitives in commodity OS kernel. In S. J. Stolfo, A. Stavrou, and C. V. Wright, editors, Research in Attacks, Intrusions, and Defenses, number 8145 in Lecture Notes in Computer Science, pages 103--122. Springer Berlin Heidelberg, Jan. 2013.
    [56]
    M. Xu, X. Jiang, R. Sandhu, and X. Zhang. Towards a VMM-based usage control framework for OS kernel integrity protection. In Proceedings of the 12th ACM Symposium on Access Control Models and Technologies, SACMAT '07, pages 71--80, New York, NY, USA, 2007. ACM.

    Cited By

    View all
    • (2024)GENESIS: A Generalizable, Efficient, and Secure Intra-kernel Privilege SeparationProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3635951(1366-1375)Online publication date: 8-Apr-2024
    • (2024)Condo: Enhancing Container Isolation Through Kernel Permission Data ProtectionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.341191519(6168-6183)Online publication date: 2024
    • (2023)Endoprocess: Programmable and Extensible Subprocess IsolationProceedings of the 2023 New Security Paradigms Workshop10.1145/3633500.3633507(92-101)Online publication date: 18-Sep-2023
    • Show More Cited By

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ASPLOS '15: Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems
    March 2015
    720 pages
    ISBN:9781450328357
    DOI:10.1145/2694344
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 14 March 2015

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. intra-kernel isolation
    2. malicious operating systems
    3. operating system architecture
    4. virtual memory

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ASPLOS '15

    Acceptance Rates

    ASPLOS '15 Paper Acceptance Rate 48 of 287 submissions, 17%;
    Overall Acceptance Rate 535 of 2,713 submissions, 20%

    Upcoming Conference

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)150
    • Downloads (Last 6 weeks)11

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)GENESIS: A Generalizable, Efficient, and Secure Intra-kernel Privilege SeparationProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3635951(1366-1375)Online publication date: 8-Apr-2024
    • (2024)Condo: Enhancing Container Isolation Through Kernel Permission Data ProtectionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.341191519(6168-6183)Online publication date: 2024
    • (2023)Endoprocess: Programmable and Extensible Subprocess IsolationProceedings of the 2023 New Security Paradigms Workshop10.1145/3633500.3633507(92-101)Online publication date: 18-Sep-2023
    • (2023)Software Compartmentalization Trade-Offs with Hardware CapabilitiesProceedings of the 12th Workshop on Programming Languages and Operating Systems10.1145/3623759.3624550(49-57)Online publication date: 23-Oct-2023
    • (2023)Veil: A Protected Services Framework for Confidential Virtual MachinesProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 410.1145/3623278.3624763(378-393)Online publication date: 25-Mar-2023
    • (2023)Creating Trust by Abolishing HierarchiesProceedings of the 19th Workshop on Hot Topics in Operating Systems10.1145/3593856.3595900(231-238)Online publication date: 22-Jun-2023
    • (2023)DynaCutProceedings of the 24th International Middleware Conference10.1145/3590140.3629121(275-287)Online publication date: 27-Nov-2023
    • (2023)Exit-Less, Isolated, and Shared Access for Virtual MachinesProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582042(224-237)Online publication date: 25-Mar-2023
    • (2023)ThreadLock: Native Principal Isolation Through Memory Protection KeysProceedings of the 2023 ACM Asia Conference on Computer and Communications Security10.1145/3579856.3595797(966-979)Online publication date: 10-Jul-2023
    • (2023)ISA-Grid: Architecture of Fine-grained Privilege Control for Instructions and RegistersProceedings of the 50th Annual International Symposium on Computer Architecture10.1145/3579371.3589050(1-15)Online publication date: 17-Jun-2023
    • Show More Cited By

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media