research-article

Iago attacks: why the system call API is a bad untrusted RPC interface

Authors:

Stephen Checkoway,

Hovav ShachamAuthors Info & Claims

ASPLOS '13: Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems

Pages 253 - 264

https://doi.org/10.1145/2451116.2451145

Published: 16 March 2013 Publication History

Abstract

In recent years, researchers have proposed systems for running trusted code on an untrusted operating system. Protection mechanisms deployed by such systems keep a malicious kernel from directly manipulating a trusted application's state. Under such systems, the application and kernel are, conceptually, peers, and the system call API defines an RPC interface between them.

We introduce Iago attacks, attacks that a malicious kernel can mount in this model. We show how a carefully chosen sequence of integer return values to Linux system calls can lead a supposedly protected process to act against its interests, and even to undertake arbitrary computation at the malicious kernel's behest.

Iago attacks are evidence that protecting applications from malicious kernels is more difficult than previously realized.

References

[1]

Anonymous. Once upon a free()łdots. Phrack Magazine, 57 (9), August 2001.http://www.phrack.org/archives/57/p57_0x09_Once%20upon%20a%20free()_by_anonymous20author.txt.

[2]

Adam Barth, Collin Jackson, Charles Reis, and The Google Chrome Team. The security architecture of the Chromium browser. Online: http://seclab.stanford.edu/websec/chromium/, 2008.

[3]

blackngel. Malloc des-maleficarum. Phrack Magazine, 66 (10), November 2009. http://www.phrack.org/archives/66/p66_0x0a_Malloc%20Des-Maleficarum_by_blackngel.txt.

[4]

blackngel. ptmalloc v2 & v3: Analysis & corruption. Phrack Magazine, 67 (8), November 2010. http://www.phrack.org/archives/67/p67_0x08_The%20House%20Of%20Lore:%20Reloaded20ptmalloc%20v2%20&%20v3:%20Analysis%20&%20Corruption_by_blackngel.txt.

[5]

Stephen Checkoway, Ariel J. Feldman, Brian Kantor, J. Alex Halderman, Edward W. Felten, and Hovav Shacham. Can DREs provide long-lasting security? The case of return-oriented programming and the AVC Advantage. In David Jefferson, Joseph Lorenzo Hall, and Tal Moran, editors, Proceedings of EVT/WOTE 2009. USENIX/ACCURATE/IAVoSS, August 2009.

Digital Library

[6]

Xiaoxin Chen, Tal Garfinkel, E. Christopher Lewis, Pratap Subrahmanyam, Carl A. Waldspurger, Dan Boneh, Jeffrey Dwoskin, and Dan R.K. Ports. Overshadow: a virtualization-based approach to retrofitting protection in commodity operating systems. In James Larus, editor, Proceedings of ASPLOS 2008, pages 2--13. ACM Press, March 2008.

Digital Library

[7]

ISO/IEC FDIS 9899:1999 (E). Programming languages -- C. ISO, 1999.

[8]

Dawson R. Engler, M. Frans Kaashoek, and James W. O'Toole. Exokernel: An operating system architecture for application-level resource management,. In Mark Weiser, editor, Proceedings of SOSP 1995, pages 251--66. ACM Press, December 1995.

Digital Library

[9]

Tal Garfinkel. Traps and pitfalls: Practical problems in system call interposition based security tools. In Virgil Gligor and Mike Reiter, editors, Proceedings of NDSS 2003. Internet Society, February 2003.

[10]

Tal Garfinkel, Ben Pfaff, and Mendel Rosenblum. Ostia: A delegating architecture for secure system call interposition. In Mike Reiter and Dan Boneh, editors, Proceedings of NDSS 2004. Internet Society, February 2004.

[11]

Ian Goldberg, David Wagner, Randi Thomas, and Eric A. Brewer. A secure environment for untrusted helper applications. In Greg Rose, editor, Proceedings of USENIX Security 1996. USENIX, July 1996.

Digital Library

[12]

David B. Golub and Richard P. Draves. Moving the default memory manager out of the mach kernel. In Alan Langerman, editor, Proceedings of Mach Symposium 1991, pages 177--88, November, 1991. USENIX.

[13]

Michel Kaempf. Vudo malloc tricks. Phrack Magazine, 57 (8), August 2001. http://www.phrack.org/archives/57/p57_0x08_Vudo%20malloc%20tricks_by_MaXX.txt.

[14]

David Lie, Chandramohan Thekkath, and Mark Horowitz. Implementing an untrusted operating system on trusted hardware. In Larry Peterson, editor, Proceedings of SOSP 2003, pages 178--92. ACM Press, October 2003.

Digital Library

[15]

Jonathan M. McCune, Bryan Parno, Adrian Perrig, Michael K. Reiter, and Arvind Seshadri. Minimal tcb code execution (extended abstract). In Birgit Pfitzmann and Patrick McDaniel, editors, Proceedings of IEEE Security & Privacy ("Oakland") 2007, pages 267--72. IEEE Computer Society, May 2007.

Digital Library

[16]

Jonathan M. McCune, Bryan Parno, Adrian Perrig, Michael K. Reiter, and Arvind Seshadri. How low can you go? Recommendations for hardware-supported minimal TCB code execution. In James Larus, editor, Proceedings of ASPLOS 2008, pages 14--25. ACM Press, March 2008.

Digital Library

[17]

Jonathan M. McCune, Bryan Parno, Adrian Perrig, Michael K. Reiter, and Arvind Seshadri. Flicker: An execution infrastructure for TCB minimization. In Steven Hand, editor, Proceedings of EuroSys 2008, pages 315--28. ACM Press, March 2008.

Digital Library

[18]

Jonathan M. McCune, Adrian Perrig, and Michael K. Reiter. Safe passage for passwords and other sensitive data. In Giovanni Vigna, editor, Proceedings of NDSS 2009. The Internet Society, February 2009.

[19]

Gene Novark and Emery D. Berger. DieHarder: Securing the heap. In Angelos D. Keromytis and Vitaly Shmatikov, editors, Proceedings of CCS 2010. ACM Press, October 2010.

Digital Library

[20]

Jon Oberheide. The stack is back. Presented at Infiltrate 2012, January 2012. Presentation. Slides: http://jon.oberheide.org/files/infiltrate12-thestackisback.pdf.

[21]

Phantasmal Phantasmagoria. The malloc maleficarum: Glibc malloc exploitation techniques. Bugtraq, October 2005. http://seclists.org/bugtraq/2005/Oct/118.

[22]

Dan R.K. Ports and Tal Garfinkel. Towards application security on untrusted operating systems. In Niels Provos, editor, Proceedings of HotSec 2008. USENIX, July 2008.

Digital Library

[23]

POSIX.1-2008/IEEE Std 1003.1-2008. The Open Group Base Specifications Issue 7. IEEE and The Open Group, 2008.

Digital Library

[24]

Niels Provos. Improving host security with system call policies. In Vern Paxson, editor, Proceedings of USENIX Security 2003. USENIX, August 2003.

Digital Library

[25]

Eric Rescorla. SSL and TLS: Designing and Building Secure Systems. Addison-Wesley, 2000.

[26]

Thomas Ristenpart and Scott Yilek. When good randomness goes bad: Virtual machine reset vulnerabilities and hedging deployed cryptography. In Wenke Lee, editor, Proceedings of NDSS 2003. Internet Society, February 2003.

[27]

Ryan Roemer, Erik Buchanan, Hovav Shacham, and Stefan Savage. Return-oriented programming: Systems, languages, and applications. Trans. Info. & Sys. Sec., 2012. To appear.

Digital Library

[28]

Alexander Sotirov and Mark Dowd. Bypassing browser memory protections in Windows Vista. Presented at Black Hat 2008, August 2008. Online: http://www.phreedom.org/research/bypassing-browser-memory-protections/bypassing-browser-memory-protections.pdf.

[29]

Scott Yilek, Eric Rescorla, Hovav Shacham, Brandon Enright, and Stefan Savage. When private keys are public: Results from the 2008 Debian OpenSSL vulnerability. In Anja Feldmann and Laurent Mathy, editors, Proceedings of IMC 2009, pages 15--27. ACM Press, November 2009.

Digital Library

Cited By

Mo FTarkhani ZHaddadi H(2024)Machine Learning with Confidential Computing: A Systematization of KnowledgeACM Computing Surveys10.1145/367000756:11(1-40)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670007
Zhang XZhang YZhang Y(2024)VeriTrain: Validating MLaaS Training Efforts via Anomaly DetectionIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.326642721:3(1032-1049)Online publication date: May-2024
https://doi.org/10.1109/TDSC.2023.3266427
Casell BColonnelli IMittone GBirke RRiviera WSciarappa ACavazzoni CAldinucci M(2024)A Performance Analysis for Confidential Federated Learning2024 IEEE Security and Privacy Workshops (SPW)10.1109/SPW63631.2024.00009(40-47)Online publication date: 23-May-2024
https://doi.org/10.1109/SPW63631.2024.00009
Show More Cited By

Index Terms

Iago attacks: why the system call API is a bad untrusted RPC interface
1. Security and privacy
  1. Intrusion/anomaly detection and malware mitigation
  2. Systems security
    1. Operating systems security

Recommendations

Iago attacks: why the system call API is a bad untrusted RPC interface
ASPLOS '13

In recent years, researchers have proposed systems for running trusted code on an untrusted operating system. Protection mechanisms deployed by such systems keep a malicious kernel from directly manipulating a trusted application's state. Under such ...
Iago attacks: why the system call API is a bad untrusted RPC interface
ASPLOS '13

In recent years, researchers have proposed systems for running trusted code on an untrusted operating system. Protection mechanisms deployed by such systems keep a malicious kernel from directly manipulating a trusted application's state. Under such ...
PESC: A Per System-Call Stack Canary Design for Linux Kernel
CODASPY '20: Proceedings of the Tenth ACM Conference on Data and Application Security and Privacy

Stack canary is the most widely deployed defense technique against stack buffer overflow attacks. However, since its proposition, the design of stack canary has very few improvements during the past 20 years, making it vulnerable to new and sophisticated ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASPLOS '13: Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems

March 2013

574 pages

ISBN:9781450318709

DOI:10.1145/2451116

General Chair:
Vivek Sarkar
Rice University, USA
,
Program Chair:
Rastislav Bodik
University of California, Berkeley, USA

ACM SIGPLAN Notices Volume 48, Issue 4
ASPLOS '13
April 2013
540 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/2499368
Issue’s Table of Contents
ACM SIGARCH Computer Architecture News Volume 41, Issue 1
ASPLOS '13
March 2013
540 pages
ISSN:0163-5964
DOI:10.1145/2490301
Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 March 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ASPLOS '13

Sponsor:

ASPLOS '13: Architectural Support for Programming Languages and Operating Systems

March 16 - 20, 2013

Texas, Houston, USA

Acceptance Rates

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

175
Total Citations
View Citations
1,579
Total Downloads

Downloads (Last 12 months)163
Downloads (Last 6 weeks)22

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mo FTarkhani ZHaddadi H(2024)Machine Learning with Confidential Computing: A Systematization of KnowledgeACM Computing Surveys10.1145/367000756:11(1-40)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670007
Zhang XZhang YZhang Y(2024)VeriTrain: Validating MLaaS Training Efforts via Anomaly DetectionIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.326642721:3(1032-1049)Online publication date: May-2024
https://doi.org/10.1109/TDSC.2023.3266427
Casell BColonnelli IMittone GBirke RRiviera WSciarappa ACavazzoni CAldinucci M(2024)A Performance Analysis for Confidential Federated Learning2024 IEEE Security and Privacy Workshops (SPW)10.1109/SPW63631.2024.00009(40-47)Online publication date: 23-May-2024
https://doi.org/10.1109/SPW63631.2024.00009
Pan DWang JChen YYu DYang WZhang Y(2024)COMURICE: Closing Source Code Leakage in Cloud-Based Compiling via Enclave2024 IEEE 11th International Conference on Cyber Security and Cloud Computing (CSCloud)10.1109/CSCloud62866.2024.00029(126-131)Online publication date: 28-Jun-2024
https://doi.org/10.1109/CSCloud62866.2024.00029
Zhang YHu YNing ZZhang FLuo XHuang HYan SHe ZCalandrino JTroncoso C(2023)SHELTERProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620587(6257-6274)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620587
He YGuo RXing YChe XSun KLiu ZXu KLi QCalandrino JTroncoso C(2023)Cross container attacksProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620571(5971-5988)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620571
Chen SLin ZZhang YCalandrino JTroncoso C(2023)Controlled data races in enclavesProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620465(4069-4086)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620465
Park JKang SLee SKim TPark JKwon YHuh J(2023)Hardware Hardened Sandbox Enclaves for Trusted Serverless ComputingACM Transactions on Architecture and Code Optimization10.1145/3632954Online publication date: 14-Nov-2023
https://dl.acm.org/doi/10.1145/3632954
Ahmad AOu BLiu CZhang XFonseca PAamodt TSwift MJerger N(2023)Veil: A Protected Services Framework for Confidential Virtual MachinesProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 410.1145/3623278.3624763(378-393)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3623278.3624763
Sarkar AKrook RRusso AClaessen KMcDonell TVazou N(2023)HasTEE: Programming Trusted Execution Environments with HaskellProceedings of the 16th ACM SIGPLAN International Haskell Symposium10.1145/3609026.3609731(72-88)Online publication date: 30-Aug-2023
https://dl.acm.org/doi/10.1145/3609026.3609731
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents