research-article

Efficient live migration of virtual machines using shared storage

Authors:

Erik Gustafsson,

Bernhard EggerAuthors Info & Claims

ACM SIGPLAN Notices, Volume 48, Issue 7

Pages 41 - 50

https://doi.org/10.1145/2517326.2451524

Published: 16 March 2013 Publication History

Abstract

Live migration of virtual machines (VM) across distinct physical hosts is an important feature of virtualization technology for maintenance, load-balancing and energy reduction, especially so for data centers operators and cluster service providers. Several techniques have been proposed to reduce the downtime of the VM being transferred, often at the expense of the total migration time. In this work, we present a technique to reduce the total time required to migrate a running VM from one host to another while keeping the downtime to a minimum. Based on the observation that modern operating systems use the better part of the physical memory to cache data from secondary storage, our technique tracks the VM's I/O operations to the network-attached storage device and maintains an updated mapping of memory pages that currently reside in identical form on the storage device. During the iterative pre-copy live migration process, instead of transferring those pages from the source to the target host, the memory-to-disk mapping is sent to the target host which then fetches the contents directly from the network-attached storage device. We have implemented our approach into the Xen hypervisor and ran a series of experiments with Linux HVM guests. On average, the presented technique shows a reduction of up over 30% on average of the total transfer time for a series of benchmarks.

References

[1]

Michael Armbrust, Armando Fox, Rean Griffith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. A view of cloud computing. Communications of the ACM, 53(4):50--58, April 2010.

Digital Library

[2]

Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield. Xen and the art of virtualization. In Proceedings of the nineteenth ACM symposium on Operating systems principles, SOSP '03, pages 164--177, New York, NY, USA, 2003. ACM.

Digital Library

[3]

Fabrice Bellard. QEMU. http://www.qemu.org, 2013. Online; accessed February 2013.

[4]

Ravi Bhargava, Benjamin Serebrin, Francesco Spadini, and Srilatha Manne. Accelerating two-dimensional page walks for virtualized systems. In Proceedings of the 13th international conference on Architectural support for programming languages and operating systems, ASPLOS XIII, pages 26--35, New York, NY, USA, 2008. ACM.

Digital Library

[5]

Nilton Bila, Eyal de Lara, Kaustubh Joshi, H. Andrés Lagar-Cavilla, Matti Hiltunen, and Mahadev Satyanarayanan. Jettison: efficient idle desktop consolidation with partial vm migration. In Proceedings of the 7th ACM european conference on Computer Systems, EuroSys '12, pages 211--224, New York, NY, USA, 2012. ACM.

Digital Library

[6]

Robert Bradford, Evangelos Kotsovinos, Anja Feldmann, and Harald Schiöberg. Live wide-area migration of virtual machines including local persistent state. In Proceedings of the 3rd international conference on Virtual execution environments, VEE '07, pages 169--179, New York, NY, USA, 2007. ACM.

Digital Library

[7]

Canonical Ltd. Ubuntu. http://www.ubuntu.com, 2013. Online; accessed February 2013.

[8]

Citrix Systems, Inc. Xen Hypervisor. http://www.xen.org/products/xenhyp.html, 2012. Online; accessed February 2013.

[9]

Christopher Clark, Keir Fraser, Steven Hand, Jacob Gorm Hansen, Eric Jul, Christian Limpach, Ian Pratt, and Andrew Warfield. Live migration of virtual machines. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2, NSDI'05, pages 273--286, Berkeley, CA, USA, 2005. USENIX Association.

Digital Library

[10]

Brendan Cully, Geoffrey Lefebvre, Dutch Meyer, Mike Feeley, Norm Hutchinson, and Andrew Warfield. Remus: high availability via asynchronous virtual machine replication. In NSDI'08: Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation, pages 161--174, Berkeley, CA, USA, 2008. USENIX Association.

Digital Library

[11]

Umesh Deshpande, Xiaoshuang Wang, and Kartik Gopalan. Live gang migration of virtual machines. In Proceedings of the 20th international symposium on High performance distributed computing, HPDC '11, pages 135--146, New York, NY, USA, 2011. ACM.

Digital Library

[12]

Irfan Habib. Virtualization with KVM. Linux Journal, 2008(166), February 2008.

Digital Library

[13]

Michael R. Hines and Kartik Gopalan. Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, VEE '09, pages 51--60, New York, NY, USA, 2009. ACM.

Digital Library

[14]

Takahiro Hirofuchi, Hidemoto Nakada, Satoshi Itoh, and Satoshi Sekiguchi. Reactive consolidation of virtual machines enabled by postcopy live migration. In Proceedings of the 5th international workshop on Virtualization technologies in distributed computing, VTDC '11, pages 11--18, New York, NY, USA, 2011. ACM.

Digital Library

[15]

Wei Huang, Qi Gao, Jiuxing Liu, and Dhabaleswar K. Panda. High performance virtual machine migration with RDMA over modern interconnects. In Proceedings of the 2007 IEEE International Conference on Cluster Computing, CLUSTER '07, pages 11--20, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[16]

Hai Jin, Li Deng, Song Wu, Xuanhua Shi, and Xiaodong Pan. Live virtual machine migration with adaptive, memory compression. In Cluster Computing and Workshops, 2009. CLUSTER '09. IEEE International Conference on, pages 1--10, 31 2009-sept. 4 2009.

[17]

Jeffrey Katcher. PostMark: A New File System Benchmark. Technical Report Technical Report TR3022, Network Appliance, October 1997.

[18]

Haikun Liu, Hai Jin, Xiaofei Liao, Liting Hu, and Chen Yu. Live migration of virtual machine based on full system trace and replay. In Proceedings of the 18th ACM international symposium on High performance distributed computing, HPDC '09, pages 101--110, New York, NY, USA, 2009. ACM.

Digital Library

[19]

Raymond A. Lorie. Physical integrity in a large segmented database. ACM Transactions on Database Systems, 2(1):91--104, March 1977.

Digital Library

[20]

Michael Nelson, Beng-Hong Lim, and Greg Hutchins. Fast transparent migration for virtual machines. In Proceedings of the annual conference on USENIX Annual Technical Conference, ATEC '05, pages 25--25, Berkeley, CA, USA, 2005. USENIX Association.

Digital Library

[21]

Oracle. VirtualBox. https://www.virtualbox.org, 2012. Online; accessed February 2013.

[22]

Eunbyung Park, Bernhard Egger, and Jaejin Lee. Fast and space-efficient virtual machine checkpointing. In Proceedings of the 7th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, VEE '11, pages 75--86, New York, NY, USA, 2011. ACM.

Digital Library

[23]

Tristan Richardson. The RFB protocol. http://www.realvnc.com/docs/rfbproto.pdf, 2010. Online; accessed February 2013.

[24]

Petter Svärd, Benoit Hudzia, Johan Tordsson, and Erik Elmroth. Evaluation of delta compression techniques for efficient live migration of large virtual machines. In Proceedings of the 7th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, VEE '11, pages 111--120, New York, NY, USA, 2011. ACM.

Digital Library

[25]

The Document Foundation. LibreOffice. http://www.libreoffice.org, 2013. Online; accessed February 2013.

[26]

The Mozilla Foundation. Firefox. http://www.mozilla.org, 2013. Online; accessed February 2013.

[27]

Franco Travostino. Seamless live migration of virtual machines over the MAN/WAN. In Proceedings of the 2006 ACM/IEEE conference on Supercomputing, SC '06, New York, NY, USA, 2006. ACM.

Digital Library

[28]

VMware. VMware VMotion: Live migration of virtual machines without service interruption. http://www.vmware.com/files/pdf/VMware-VMotion-DS-EN.pdf, 2009. Online; accessed February 2013.

Cited By

Eswaran RYan MGopalan K(2024)Tackling Memory Footprint Expansion During Live Migration of Virtual Machines2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid59990.2024.00027(158-167)Online publication date: 6-May-2024
https://doi.org/10.1109/CCGrid59990.2024.00027
Haris RBarhamgi MNhlabatsi AKhan K(2024)Optimizing pre-copy live virtual machine migration in cloud computing using machine learning-based prediction modelComputing10.1007/s00607-024-01318-6106:9(3031-3062)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1007/s00607-024-01318-6
Bhagat SGupta P(2022)A Survey on Scalable Resource Allocation in Cloud ComputingRecent Innovations in Computing10.1007/978-981-16-8248-3_33(401-414)Online publication date: 10-Mar-2022
https://doi.org/10.1007/978-981-16-8248-3_33
Show More Cited By

Index Terms

Efficient live migration of virtual machines using shared storage
1. Networks
  1. Network protocols
2. Software and its engineering
  1. Software organization and properties
    1. Extra-functional properties
      1. Software fault tolerance
        Checkpoint / restart

Recommendations

Efficient live migration of virtual machines using shared storage
VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

Live migration of virtual machines (VM) across distinct physical hosts is an important feature of virtualization technology for maintenance, load-balancing and energy reduction, especially so for data centers operators and cluster service providers. ...
Traffic-sensitive live migration of virtual machines
CCGRID '15: Proceedings of the 15th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing

In this paper we address the problem of network contention between the migration traffic and the Virtual Machine (VM) application traffic for the live migration of co-located Virtual Machines. When VMs are migrated with pre-copy, they run at the source ...
A live storage migration mechanism over wan and its performance evaluation
VTDC '09: Proceedings of the 3rd international workshop on Virtualization technologies in distributed computing

Live migration of virtual machines is a key technology for the next generation of IaaS cloud services, contributing to dynamic portability and mobility of VM-based services among datacenters. The practical use of live migration, however, is still ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 48, Issue 7

VEE '13

July 2013

194 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/2517326

Issue’s Table of Contents

VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments
March 2013
210 pages
ISBN:9781450312660
DOI:10.1145/2451512
General Chair:
Steve Muir
VMware, USA
,
Program Chairs:
Gernot Heiser
NICTA and University of New South Wales, Australia
,
Steve Blackburn
Australian National University, Australia

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 March 2013

Published in SIGPLAN Volume 48, Issue 7

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

78
Total Citations
View Citations
950
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)2

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Eswaran RYan MGopalan K(2024)Tackling Memory Footprint Expansion During Live Migration of Virtual Machines2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid59990.2024.00027(158-167)Online publication date: 6-May-2024
https://doi.org/10.1109/CCGrid59990.2024.00027
Haris RBarhamgi MNhlabatsi AKhan K(2024)Optimizing pre-copy live virtual machine migration in cloud computing using machine learning-based prediction modelComputing10.1007/s00607-024-01318-6106:9(3031-3062)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1007/s00607-024-01318-6
Bhagat SGupta P(2022)A Survey on Scalable Resource Allocation in Cloud ComputingRecent Innovations in Computing10.1007/978-981-16-8248-3_33(401-414)Online publication date: 10-Mar-2022
https://doi.org/10.1007/978-981-16-8248-3_33
Motaki SYahyaouy AGualous H(2021)A prediction-based model for virtual machine live migration monitoring in a cloud datacenterComputing10.1007/s00607-021-00981-3103:11(2711-2735)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1007/s00607-021-00981-3
Park HLee MHong C(2020)FirepanIF: High Performance Host-Side Flash Cache Warm-Up Method in Cloud ComputingApplied Sciences10.3390/app1003101410:3(1014)Online publication date: 4-Feb-2020
https://doi.org/10.3390/app10031014
Le T(2020)A survey of live Virtual Machine migration techniquesComputer Science Review10.1016/j.cosrev.2020.10030438(100304)Online publication date: Nov-2020
https://doi.org/10.1016/j.cosrev.2020.100304
Tyj NG V(2019)Adaptive deduplication of virtual machine images using AKKA stream to accelerate live migration process in cloud environmentJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-019-0125-z8:1(1-12)Online publication date: 1-Dec-2019
https://dl.acm.org/doi/10.1186/s13677-019-0125-z
Li CFeng DHua YQin L(2019)Efficient live virtual machine migration for memory write-intensive workloadsFuture Generation Computer Systems10.1016/j.future.2018.12.048Online publication date: Jan-2019
https://doi.org/10.1016/j.future.2018.12.048
Nayyer MRaza IHussain S(2019)Revisiting VM performance and optimization challenges for big data10.1016/bs.adcom.2019.02.002Online publication date: 2019
https://doi.org/10.1016/bs.adcom.2019.02.002
Terra-Neves MLynce IManquinho V(2019)Virtual machine consolidation using constraint-based multi-objective optimizationJournal of Heuristics10.1007/s10732-018-9400-225:3(339-375)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s10732-018-9400-2
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents