Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

LinuxCon Japan 2010 suzaki

•
•
Kuniyasu Suzaki
Kuniyasu Suzaki

Titie: Rethink Package Components on De-Duplication: From Logical Sharing to Physical Sharing URL: http://events.linuxfoundation.org/2010/linuxcon-japan/suzaki Abstract: inux distributions include many logical sharing techniques (shared library, symbolic link, etc) on memory and storage. Unfortunately they cause security and management problems, such as GOT (Global Offset Table) overwrite attack, TOCTTOU (Time Of Check To Time Of Use) attack, Dependency hell, etc. In order to mitigate the problems, we propose the replacement of logical sharing by physical sharing (memory and disk deduplication; e.g., KSM: Kernel Samepage Merging, Content Addressable Storage, etc). Original ELF binaries are transformed as self-contained binaries which include dynamic linked shared libraries as “pseudo-static”. The binaries become fat but physical resource usage is mitigated by deduplication. We have investigated the effect on Debian and Ubuntu and confirmed that the physical impact is low. Data centers of Cloud Computing utilize the deduplication techniques. Users and administrators should consider Linux images in terms of security and maintenance, and the usage of deduplicated fat binaries.

1 of 24
Rethink Package Components on De-Duplication: From Logical Sharing to Physical Sharing Leaflet is wrong Kuniyashu Kuniyasu Suzaki, T hiki Y i K K i h K i S ki Toshiki Yagi, Kengo Iiji Iijima, N Nguyen Anh Quynh, C ill A h A hQ h Cyrille Artho Research Center of Information Security National Institute of Advanced Industrial Science and Technology & Yoshihito Watanebe Alpha Systems Inc.
Contents • Rethink Package Components • Vulnerability of logical sharing (Dynamic-Link Shared Library and Symbolic Link) • Propose replacement of logical sharing by physical sharing – Physical sharing • Deduplication on Memory and Storage – Self-contained binary • It is NOT static-Link binary. • Experimental results • Conclusions with discussing topics
Rethink Package Components • Current Packages depend on Logical Sharing • Logical sharing is OS technique to reduce consumption of memory and storage. – “Dynamic-Link Shared Library” for memory and storage – “Symbolic Link” for storage Symbolic Link • Unfortunately, they include vulnerability caused by dynamic management. – Search Path Replacement Attack – GOT (Global Offset Table) overwrite attack – Dependency Hell – Etc.
Search Path Replacement Attack • Dynamic-link searches a shared library at run time using a search path. – Search path is defined by environment variables. • Example: “LD_LIBRARY_PATH” – It allows us to change shared libraries for each process. • Unfortunately, the search path is easily replaced by an attacker and leads to malicious shared libraries. – Caller program has no methods to certify libraries. • Static-link solves this problem but it wastes memory and storage.
GOT Overwrite Attack • ELF format has GOT (Global Offset Table) to locate position-independent function address of shared library. The value of GOT is assigned at run time. – GOT is created on Data Segment and vulnerable for overwrite attack. • Static link solves this problem but it wastes memory and storage. Program Library Call Routine Code Code Segment Segment PLT PLT Data Data GOT GOT Segment Segment Attack
Dependency Hell (DLL Hell in Windows) • Dependency Hell is a management problem of shared libraries. – Package manager maintains versions of libraries. However, the version mismatch may occur, when a user updates a library without package manager. • D Dependency H ll i escalated b symbolic-link, b d Hell is l t d by b li li k because most shared libraries use symbolic-link to manage minor updates. – /lib/libc.so.6 -> libc-2.10.1.so – # ln –s libc-2.11.1.so libc.so.6 • Static link solves this problem but it wastes memory and storage.
Solution, and further problems • The problems are solved by static-link, but it increases consumption of memory and storage. – Fortunately, the increased consumption is mitigated by new technique, deduplication. – SLINY[USENIX’05] developed deduplication in Linux kernel. – It looks the problems are solved … • Two trends – Current applications assume dynamic-link and are not re-compiled as static-link easily . – Current virtualization offers us deduplication. • SLINKY uses special Linux kernel. It is not applied on any Linux. • Using virtualization, guest OS only has to solve the security problems without regard to physical consumption.
Static-Link is not easy [BIG Problem] • Current applications depend on dynamic-link shared libraries for flexibility and avoiding license contamination. – Some Licenses conflict and the code can not be mixed. – Static link is not created with compile option, because dlopen() which calls shared library explicitly is used in main source codes. • glibc, l b l b libX11, openssl, gcc, glib, pam… Difficult to remove! l lb iffi l ! • We tried to re-compile /bin, /sbin, /usr/bin, and /usr/sbin dynamic-linked binaries (1,162) with static-link on Gentoo. – 185 (15.9%) binaries are re-compiled with static-link. • Binary packages make it difficult to re-compile, because they are not easy to get all source code. – Commercial applications make problem more difficult.
Self-Contained Binaries • Self-contained binary translator • It is developed to bring a binary to another machine. • It integrates shared libraries into an ELF binary file. – Tools • Statifier, Autopacage, Ermine for Linux – Advantage • Prevent Search Path Replacement Attack and Dependency Hell, because it integrates all libraries. • Mitigate GOT Overwrite Attack, because the addresses are prefixed for each execution. – Disadvantage • Consume more memory and storage than static-link
Min: /usr/bin/qmake Max: /usr/bin/gnome-open (3,426,340 -> 6,094,848 (x1.78) ) (5,400 -> 8,732,672 (x1617.16)) 6 shared libraries 22shared libraries # ldd /usr/bin/qmake # ldd /usr/bin/gnome-open linux-gate.so.1 => (0xb78d5000) l inux-gate.so.1 => (0xb772d000) libstdc++.so.6 => /usr/lib/gcc/i686-pc-linux-gnu/ libgnome-2.so.0 => /usr/lib/libgnome-2.so.0 (0xb7709000) 4.3.4/libstdc++.so.6 (0xb77da000) libgnomevfs-2.so.0 => /usr/lib/libgnomevfs-2.so.0 (0xb76ae000) libm.so.6 => /lib/libm.so.6 (0xb77b4000) libxml2.so.2 => /usr/lib/libxml2.so.2 (0xb7578000) libgcc_s.so.1 => /usr/lib/gcc/i686-pc-linux-gnu/ libm.so.6 => /lib/libm.so.6 (0xb7552000) 4.3.4/libgcc_s.so.1 (0xb77a6000) libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0xb7509000) libc.so.6 => /lib/libc.so.6 (0xb765e000) libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 (0xb73bc000) /lib/ld-linux.so.2 (0xb78d6000) libz.so.1 => /lib/libz.so.1 (0xb73a8000) libutil.so.1 => /lib/libutil.so.1 (0xb73a4000) libbonobo-2.so.0 => /usr/lib/libbonobo-2.so.0 (0xb7348000) libbonobo-activation.so.4 => /usr/lib/libbonobo-activation.so.4 (0xb7332000) libORBitCosNaming 2.so.0 /usr/lib/libORBitCosNaming 2.so.0 libORBitCosNaming-2 so 0 => /usr/lib/libORBitCosNaming-2 so 0 (0xb732c000) libgconf-2.so.4 => /usr/lib/libgconf-2.so.4 (0xb72f7000) libORBit-2.so.0 => /usr/lib/libORBit-2.so.0 (0xb72a2000) libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0 (0xb729c000) libpthread.so.0 => /lib/libpthread.so.0 (0xb7283000) librt.so.1 => /lib/librt.so.1 (0xb727a000) libdbus-glib-1.so.2 => /usr/lib/libdbus-glib-1.so.2 (0xb725c000) libnsl.so.1 => /lib/libnsl.so.1 (0xb7244000) libdbus-1.so.3 => /usr/lib/libdbus-1.so.3 (0xb720b000) libgio-2.0.so.0 => /usr/lib/libgio-2.0.so.0 (0xb7178000) libresolv.so.2 => /lib/libresolv.so.2 (0xb7163000) libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0xb7126000) libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0xb7120000) libdl.so.2 => /lib/libdl.so.2 (0xb711c000) libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0xb7044000) libpopt.so.0 => /usr/lib/libpopt.so.0 (0xb7039000) libc.so.6 => /lib/libc.so.6 (0xb6ef1000) /lib/ld-linux.so.2 (0xb772e000)
Statifier (1/2) • Creation of self-contained binary by Statifier • Statifier emulates normal loader and gets allocation information – Take snapshot before _dl_start_user() and analyze allocation information of functions of libraries from /proc/PID/maps. – The libraries and allocation information are embedded into the binary.
Statifier (2/2) • Self-Contained Binary – Allocation information and shared libraries are loaded by the starter which is embedded ELF binary by statifier. • Includes special libraries: linux-gate.so, ld-linux.so – The ELF binary has no INTERP segment to call ld-linux.so – ldd command shows no dynamic-link shared libraries • Statifier makes a larger binary than static link. • Increased memory and storage are mitigated by deduplication.
Deduplication • Technique to share same-content chunks at block level (memory and storage). • Same-content chunks are shared by indirect link. – It is easy to implement when a virtual layer exists to access a block device device. – Some virtualizations include deduplication mechanism.
Storage Deduplication • Used by CAS (Content addressable Storage) – data is not addressed by its physical location. Data is addressed by a unique name derived from the content (a secure hash is used as a unique name usually) – Same contents are expressed by one original content (same hash) and addressed by indirected link. • Plan9 has Venti [USENIX FAST02] • Data Domain (EMC) Deduplication [USENIX FAST08] • LBCAS (Loopback Content Addressable Storage) [LinuxSymp09] Virtual Disk CAS Storage Archive Indexing Address SHA-1 0000000-0003FFF 4ad36ffe8… New block is 0004000-0007FFF 974daf34a… created with 0008000-000BFFF 2d34ff3e1… new SHA-1 000C000-000FFFF 974daf34a… … … sharing Deduplication
Memory Deduplication • Memory deduplication is mainly used for virtual machines. • Very effective when same guest OS runs on several virtual machines. • On Virtual Machine Monitor – Disco[OSDI97] has Transparent Page Sharing – VMWare ESX has Content-Based Page Sharing [SOSP02] – Xen has Satori[USENIX09] and Differential Engine[OSDI08] • On Kernel – Linux has KSM (Kernel Samepage Merging) from 2.6.32 [LinuxSymp09] Guest Physical Memory VM1 VM2 VM(n) • Memory of Process(es) are deduplicated • KVM uses this mechanism • These targets are virtual machines, but our proposal uses memory deduplication on single OS image, which increase same pages with copy of libraries (self-contained binary). Real Physical Memory
Evaluation • Evaluate the effect of moving form logical sharing to physical sharing. – Effect of dynamic link shared library (logical sharing) • Gentoo installed on 32GB virtual disk for KVM virtual machine – Effect of Statifier (remove logical sharing and increase consumption on memory and storage) • Applied on binaries under /bin,/sbin,/usr/bin,/usr/sbin – Memory Deduplication • KSM (Kernel Samepage merging) of Linux with KVM virtual machine (758MB). – Storage Deduplication • LBCAS (Loopback Content Addressable Storage)
Effect of Dynamic Link Shared Library • 42 normal processes were running at the end of login. • Shared libraries overlapped on physical memory – (total Virtual memory used by library) / (Physical memory Used by library) = ((B) - (C) )/ ((A) - (C)) = 86,184KB / 13,016KB = 6.62 • (A) Consumed physical memory 54,760 KB • (B) Summation of consumed memory by each process (Size on /proc/[pid]/smaps) 127,928 KB • (C) Summation of process own memory 41,744KB • The overlap was 6.62 times. Information obtained by /proc/[pid]smaps Number of libraries used by 42 processes PID RSS SHARED Usued Name 1 672 536 (79%) 42 linux-gate.so.1 Included by 792 1676 448 (26%) 42 /lib/ld-linux.so.2 all processes 1401 508 400 (78%) 42 libc.so.6 1412 712 580 (81%) 32 libdl.so.2 1709 544 392 (72%) 29 libpthread.so.0 1718 408 332 (81%) 27 libpcre.so.3 1728 628 524 (83%) 27 libnsl.so.1 1737 764 360 (47%) 26 libglib-2.0.so.0 1749 1428 1040 (72%) 25 libgobject-2.0.so.0 1750 468 372 (79%) 24 libm.so.6
Static Analysis of Statifier • Gentoo was customized by statifier. – The ELF (1,162) binaries under /bin (82 files), /sbin (74), /usr/bin (912), /usr/sbin (94) were customized by statifier. Original Statifier Increase (Dynamic-link) Total 87,865,480 3,572,936,704 40.66 Average 75,615 3,074,816 40.66 Max (gnome-open) 5,400 8,732,672 1617.16 Min (qmake) 3,426,340 6,094,848 1.78 • The disk image (includes non-statifiered files) was expnaded from 3.75GB to 7.08GB (1.88 times).
Effect of Memory Deduplication • Memory usage at the end of login • Statifier expanded memory consumption from the view of GuestOS, • but Deduplication reduced physical memory consumption. 80000 34.4% 4KB GuestOS View page 70000 60000 50000 GuestOS 93.0% 45 332 View Duplicated 40000 86056 8.9% Deduplicate 22 96 17.3% 30000 48 1 44 41 29732 Unique physical 20000 32706 30410 memory 2 9929 physical 25 291 10000 memory 0 Normal Gentoo Statifier Gentoo
Trace of memory consumption Normal Gentoo Statifer Gentoo Loopback GuestOS View Physical Physical Mem View Mem View GuestOS View Auto Auto login login LBCAS (256KB) GuestOS View Physical GuestOS View Mem View Physical Mem View Auto Auto login login
Effect of Storage Deduplication • Storage usage (static) and total read data at boot (dynamic) . • Statifier expanded storage consumption from the view of GuestOS on both cases, but Deduplication reduced physical storage consumption in static and dynamic. • Smaller chunk is easy to be deduplicated but time overhead is large. Static Dynamic ( (boot) ) normal statifier normal statifier On Loopback 3,754MB 7,075MB 151.7MB 341.0MB (Guest OS View) (1.88) (2.25) 4352MB 268,454 LBCAS 16KB [278,499] --- ---- [4195MB] (1.04) 83,863 304MB 74,679 218MB LBCAS 64KB [5241MB] [4,866] [4667MB] [3,481] (1.12) (1.40) 6723MB 505MB 22,806 390MB LBCAS 256KB [26,892] [2,019] [5701MB] [1,560 ] (1.18) (1.29)
Time overhead at boot • Statifier reduced the boot time, because it eliminated dynamic reallocation overhead. • Deduplication increased the boot time. The overhead of KSM and LBCAS was less than 37%. – The overhead is a penalty to remove the vulnerabilities of logical sharing. Without KSM With KSM Normal Statifier Normal Statifier Loopback 95s 84s 95s 105s Reduced LBCAS 107s 108s 115s 130s (256KB)
Conclusion & Discussion • Self-Contained binaries strengthen OS security. – Prevent Search Path Replacement Attack, GOT (Global Offset Table) overwrite attack, Dependency Hell – Easy to apply on normal OS. It does not require source code and re-compile. – Increase consumption of memory and storage. • Deduplication mitigates the consumption of memory and storage caused by self-contained binary. – Encourage moving from Logical sharing to Physical Sharing • Deduplication is utilized to increase security on single OS. ＊Technical detail is written in USENIX HotSec2010 Paper, “Moving from Logical Sharing of Guest OS to Physical Sharing of Deduplication on Virtual Machine”
Conclusion & Discussion • Re-think the packages of Linux distributions • Self-Contained binaries is temporary solution. We should re- think to remove logical sharing which cause security problems, because same contents are shared on memory and storage p dedeuplcation. • Deduplication will be mainly used on IaaS type (multi-tenants) Cloud Computing. ＊Technical detail is written in USENIX HotSec2010 Paper, “Moving from Logical Sharing of Guest OS to Physical Sharing of Deduplication on Virtual Machine”

More Related Content

LinuxCon Japan 2010 suzaki

  • 1. Rethink Package Components on De-Duplication: From Logical Sharing to Physical Sharing Leaflet is wrong Kuniyashu Kuniyasu Suzaki, T hiki Y i K K i h K i S ki Toshiki Yagi, Kengo Iiji Iijima, N Nguyen Anh Quynh, C ill A h A hQ h Cyrille Artho Research Center of Information Security National Institute of Advanced Industrial Science and Technology & Yoshihito Watanebe Alpha Systems Inc.
  • 2. Contents • Rethink Package Components • Vulnerability of logical sharing (Dynamic-Link Shared Library and Symbolic Link) • Propose replacement of logical sharing by physical sharing – Physical sharing • Deduplication on Memory and Storage – Self-contained binary • It is NOT static-Link binary. • Experimental results • Conclusions with discussing topics
  • 3. Rethink Package Components • Current Packages depend on Logical Sharing • Logical sharing is OS technique to reduce consumption of memory and storage. – “Dynamic-Link Shared Library” for memory and storage – “Symbolic Link” for storage Symbolic Link • Unfortunately, they include vulnerability caused by dynamic management. – Search Path Replacement Attack – GOT (Global Offset Table) overwrite attack – Dependency Hell – Etc.
  • 4. Search Path Replacement Attack • Dynamic-link searches a shared library at run time using a search path. – Search path is defined by environment variables. • Example: “LD_LIBRARY_PATH” – It allows us to change shared libraries for each process. • Unfortunately, the search path is easily replaced by an attacker and leads to malicious shared libraries. – Caller program has no methods to certify libraries. • Static-link solves this problem but it wastes memory and storage.
  • 5. GOT Overwrite Attack • ELF format has GOT (Global Offset Table) to locate position-independent function address of shared library. The value of GOT is assigned at run time. – GOT is created on Data Segment and vulnerable for overwrite attack. • Static link solves this problem but it wastes memory and storage. Program Library Call Routine Code Code Segment Segment PLT PLT Data Data GOT GOT Segment Segment Attack
  • 6. Dependency Hell (DLL Hell in Windows) • Dependency Hell is a management problem of shared libraries. – Package manager maintains versions of libraries. However, the version mismatch may occur, when a user updates a library without package manager. • D Dependency H ll i escalated b symbolic-link, b d Hell is l t d by b li li k because most shared libraries use symbolic-link to manage minor updates. – /lib/libc.so.6 -> libc-2.10.1.so – # ln –s libc-2.11.1.so libc.so.6 • Static link solves this problem but it wastes memory and storage.
  • 7. Solution, and further problems • The problems are solved by static-link, but it increases consumption of memory and storage. – Fortunately, the increased consumption is mitigated by new technique, deduplication. – SLINY[USENIX’05] developed deduplication in Linux kernel. – It looks the problems are solved … • Two trends – Current applications assume dynamic-link and are not re-compiled as static-link easily . – Current virtualization offers us deduplication. • SLINKY uses special Linux kernel. It is not applied on any Linux. • Using virtualization, guest OS only has to solve the security problems without regard to physical consumption.
  • 8. Static-Link is not easy [BIG Problem] • Current applications depend on dynamic-link shared libraries for flexibility and avoiding license contamination. – Some Licenses conflict and the code can not be mixed. – Static link is not created with compile option, because dlopen() which calls shared library explicitly is used in main source codes. • glibc, l b l b libX11, openssl, gcc, glib, pam… Difficult to remove! l lb iffi l ! • We tried to re-compile /bin, /sbin, /usr/bin, and /usr/sbin dynamic-linked binaries (1,162) with static-link on Gentoo. – 185 (15.9%) binaries are re-compiled with static-link. • Binary packages make it difficult to re-compile, because they are not easy to get all source code. – Commercial applications make problem more difficult.
  • 9. Self-Contained Binaries • Self-contained binary translator • It is developed to bring a binary to another machine. • It integrates shared libraries into an ELF binary file. – Tools • Statifier, Autopacage, Ermine for Linux – Advantage • Prevent Search Path Replacement Attack and Dependency Hell, because it integrates all libraries. • Mitigate GOT Overwrite Attack, because the addresses are prefixed for each execution. – Disadvantage • Consume more memory and storage than static-link
  • 10. Min: /usr/bin/qmake Max: /usr/bin/gnome-open (3,426,340 -> 6,094,848 (x1.78) ) (5,400 -> 8,732,672 (x1617.16)) 6 shared libraries 22shared libraries # ldd /usr/bin/qmake # ldd /usr/bin/gnome-open linux-gate.so.1 => (0xb78d5000) l inux-gate.so.1 => (0xb772d000) libstdc++.so.6 => /usr/lib/gcc/i686-pc-linux-gnu/ libgnome-2.so.0 => /usr/lib/libgnome-2.so.0 (0xb7709000) 4.3.4/libstdc++.so.6 (0xb77da000) libgnomevfs-2.so.0 => /usr/lib/libgnomevfs-2.so.0 (0xb76ae000) libm.so.6 => /lib/libm.so.6 (0xb77b4000) libxml2.so.2 => /usr/lib/libxml2.so.2 (0xb7578000) libgcc_s.so.1 => /usr/lib/gcc/i686-pc-linux-gnu/ libm.so.6 => /lib/libm.so.6 (0xb7552000) 4.3.4/libgcc_s.so.1 (0xb77a6000) libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0xb7509000) libc.so.6 => /lib/libc.so.6 (0xb765e000) libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 (0xb73bc000) /lib/ld-linux.so.2 (0xb78d6000) libz.so.1 => /lib/libz.so.1 (0xb73a8000) libutil.so.1 => /lib/libutil.so.1 (0xb73a4000) libbonobo-2.so.0 => /usr/lib/libbonobo-2.so.0 (0xb7348000) libbonobo-activation.so.4 => /usr/lib/libbonobo-activation.so.4 (0xb7332000) libORBitCosNaming 2.so.0 /usr/lib/libORBitCosNaming 2.so.0 libORBitCosNaming-2 so 0 => /usr/lib/libORBitCosNaming-2 so 0 (0xb732c000) libgconf-2.so.4 => /usr/lib/libgconf-2.so.4 (0xb72f7000) libORBit-2.so.0 => /usr/lib/libORBit-2.so.0 (0xb72a2000) libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0 (0xb729c000) libpthread.so.0 => /lib/libpthread.so.0 (0xb7283000) librt.so.1 => /lib/librt.so.1 (0xb727a000) libdbus-glib-1.so.2 => /usr/lib/libdbus-glib-1.so.2 (0xb725c000) libnsl.so.1 => /lib/libnsl.so.1 (0xb7244000) libdbus-1.so.3 => /usr/lib/libdbus-1.so.3 (0xb720b000) libgio-2.0.so.0 => /usr/lib/libgio-2.0.so.0 (0xb7178000) libresolv.so.2 => /lib/libresolv.so.2 (0xb7163000) libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0xb7126000) libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0xb7120000) libdl.so.2 => /lib/libdl.so.2 (0xb711c000) libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0xb7044000) libpopt.so.0 => /usr/lib/libpopt.so.0 (0xb7039000) libc.so.6 => /lib/libc.so.6 (0xb6ef1000) /lib/ld-linux.so.2 (0xb772e000)
  • 11. Statifier (1/2) • Creation of self-contained binary by Statifier • Statifier emulates normal loader and gets allocation information – Take snapshot before _dl_start_user() and analyze allocation information of functions of libraries from /proc/PID/maps. – The libraries and allocation information are embedded into the binary.
  • 12. Statifier (2/2) • Self-Contained Binary – Allocation information and shared libraries are loaded by the starter which is embedded ELF binary by statifier. • Includes special libraries: linux-gate.so, ld-linux.so – The ELF binary has no INTERP segment to call ld-linux.so – ldd command shows no dynamic-link shared libraries • Statifier makes a larger binary than static link. • Increased memory and storage are mitigated by deduplication.
  • 13. Deduplication • Technique to share same-content chunks at block level (memory and storage). • Same-content chunks are shared by indirect link. – It is easy to implement when a virtual layer exists to access a block device device. – Some virtualizations include deduplication mechanism.
  • 14. Storage Deduplication • Used by CAS (Content addressable Storage) – data is not addressed by its physical location. Data is addressed by a unique name derived from the content (a secure hash is used as a unique name usually) – Same contents are expressed by one original content (same hash) and addressed by indirected link. • Plan9 has Venti [USENIX FAST02] • Data Domain (EMC) Deduplication [USENIX FAST08] • LBCAS (Loopback Content Addressable Storage) [LinuxSymp09] Virtual Disk CAS Storage Archive Indexing Address SHA-1 0000000-0003FFF 4ad36ffe8… New block is 0004000-0007FFF 974daf34a… created with 0008000-000BFFF 2d34ff3e1… new SHA-1 000C000-000FFFF 974daf34a… … … sharing Deduplication
  • 15. Memory Deduplication • Memory deduplication is mainly used for virtual machines. • Very effective when same guest OS runs on several virtual machines. • On Virtual Machine Monitor – Disco[OSDI97] has Transparent Page Sharing – VMWare ESX has Content-Based Page Sharing [SOSP02] – Xen has Satori[USENIX09] and Differential Engine[OSDI08] • On Kernel – Linux has KSM (Kernel Samepage Merging) from 2.6.32 [LinuxSymp09] Guest Physical Memory VM1 VM2 VM(n) • Memory of Process(es) are deduplicated • KVM uses this mechanism • These targets are virtual machines, but our proposal uses memory deduplication on single OS image, which increase same pages with copy of libraries (self-contained binary). Real Physical Memory
  • 16. Evaluation • Evaluate the effect of moving form logical sharing to physical sharing. – Effect of dynamic link shared library (logical sharing) • Gentoo installed on 32GB virtual disk for KVM virtual machine – Effect of Statifier (remove logical sharing and increase consumption on memory and storage) • Applied on binaries under /bin,/sbin,/usr/bin,/usr/sbin – Memory Deduplication • KSM (Kernel Samepage merging) of Linux with KVM virtual machine (758MB). – Storage Deduplication • LBCAS (Loopback Content Addressable Storage)
  • 17. Effect of Dynamic Link Shared Library • 42 normal processes were running at the end of login. • Shared libraries overlapped on physical memory – (total Virtual memory used by library) / (Physical memory Used by library) = ((B) - (C) )/ ((A) - (C)) = 86,184KB / 13,016KB = 6.62 • (A) Consumed physical memory 54,760 KB • (B) Summation of consumed memory by each process (Size on /proc/[pid]/smaps) 127,928 KB • (C) Summation of process own memory 41,744KB • The overlap was 6.62 times. Information obtained by /proc/[pid]smaps Number of libraries used by 42 processes PID RSS SHARED Usued Name 1 672 536 (79%) 42 linux-gate.so.1 Included by 792 1676 448 (26%) 42 /lib/ld-linux.so.2 all processes 1401 508 400 (78%) 42 libc.so.6 1412 712 580 (81%) 32 libdl.so.2 1709 544 392 (72%) 29 libpthread.so.0 1718 408 332 (81%) 27 libpcre.so.3 1728 628 524 (83%) 27 libnsl.so.1 1737 764 360 (47%) 26 libglib-2.0.so.0 1749 1428 1040 (72%) 25 libgobject-2.0.so.0 1750 468 372 (79%) 24 libm.so.6
  • 18. Static Analysis of Statifier • Gentoo was customized by statifier. – The ELF (1,162) binaries under /bin (82 files), /sbin (74), /usr/bin (912), /usr/sbin (94) were customized by statifier. Original Statifier Increase (Dynamic-link) Total 87,865,480 3,572,936,704 40.66 Average 75,615 3,074,816 40.66 Max (gnome-open) 5,400 8,732,672 1617.16 Min (qmake) 3,426,340 6,094,848 1.78 • The disk image (includes non-statifiered files) was expnaded from 3.75GB to 7.08GB (1.88 times).
  • 19. Effect of Memory Deduplication • Memory usage at the end of login • Statifier expanded memory consumption from the view of GuestOS, • but Deduplication reduced physical memory consumption. 80000 34.4% 4KB GuestOS View page 70000 60000 50000 GuestOS 93.0% 45 332 View Duplicated 40000 86056 8.9% Deduplicate 22 96 17.3% 30000 48 1 44 41 29732 Unique physical 20000 32706 30410 memory 2 9929 physical 25 291 10000 memory 0 Normal Gentoo Statifier Gentoo
  • 20. Trace of memory consumption Normal Gentoo Statifer Gentoo Loopback GuestOS View Physical Physical Mem View Mem View GuestOS View Auto Auto login login LBCAS (256KB) GuestOS View Physical GuestOS View Mem View Physical Mem View Auto Auto login login
  • 21. Effect of Storage Deduplication • Storage usage (static) and total read data at boot (dynamic) . • Statifier expanded storage consumption from the view of GuestOS on both cases, but Deduplication reduced physical storage consumption in static and dynamic. • Smaller chunk is easy to be deduplicated but time overhead is large. Static Dynamic ( (boot) ) normal statifier normal statifier On Loopback 3,754MB 7,075MB 151.7MB 341.0MB (Guest OS View) (1.88) (2.25) 4352MB 268,454 LBCAS 16KB [278,499] --- ---- [4195MB] (1.04) 83,863 304MB 74,679 218MB LBCAS 64KB [5241MB] [4,866] [4667MB] [3,481] (1.12) (1.40) 6723MB 505MB 22,806 390MB LBCAS 256KB [26,892] [2,019] [5701MB] [1,560 ] (1.18) (1.29)
  • 22. Time overhead at boot • Statifier reduced the boot time, because it eliminated dynamic reallocation overhead. • Deduplication increased the boot time. The overhead of KSM and LBCAS was less than 37%. – The overhead is a penalty to remove the vulnerabilities of logical sharing. Without KSM With KSM Normal Statifier Normal Statifier Loopback 95s 84s 95s 105s Reduced LBCAS 107s 108s 115s 130s (256KB)
  • 23. Conclusion & Discussion • Self-Contained binaries strengthen OS security. – Prevent Search Path Replacement Attack, GOT (Global Offset Table) overwrite attack, Dependency Hell – Easy to apply on normal OS. It does not require source code and re-compile. – Increase consumption of memory and storage. • Deduplication mitigates the consumption of memory and storage caused by self-contained binary. – Encourage moving from Logical sharing to Physical Sharing • Deduplication is utilized to increase security on single OS. ＊Technical detail is written in USENIX HotSec2010 Paper, “Moving from Logical Sharing of Guest OS to Physical Sharing of Deduplication on Virtual Machine”
  • 24. Conclusion & Discussion • Re-think the packages of Linux distributions • Self-Contained binaries is temporary solution. We should re- think to remove logical sharing which cause security problems, because same contents are shared on memory and storage p dedeuplcation. • Deduplication will be mainly used on IaaS type (multi-tenants) Cloud Computing. ＊Technical detail is written in USENIX HotSec2010 Paper, “Moving from Logical Sharing of Guest OS to Physical Sharing of Deduplication on Virtual Machine”