Optimized P2V conversion using VMware Converter Standalone

Optimized P2V conversion using VMware Converter Standalone Ionel Gordin “Stefan cel Mare” University Suceava, Romania ionel@usv.ro Abstract — Server virtualization it is a gamechanging technology for IT, providing efficiencies and capabilities that just aren't possible when constrained within a physical world. Even that server virtualization has continued to mature and advance itself, some organizations are still not taking full advantages of the virtualization. An organization with a lot of real servers will have to allocate a lot of time for transitioning process to virtualized servers. The purpose of this article is to analyze and provide the best procedures for optimizing the speed of the conversion process using VMware Converter. We have performed tests for Linux and Windows operating systems using different hardware configurations. Concrete results and conclusions were provided for encrypted/non encrypted data transfers respectively using multiple data streams during virtualization process. conversion, the resulting virtual machine will not be an exact clone of the source system. Converter Standalone can shut down the source machine and power on the destination machine when the conversion process is complete. When combined with synchronization, this action allows complete migration of a source machine to a virtual machine destination. The destination machine takes over the source machine operations with the least possible downtime. We will be concentrating this article on optimizing performance of hot cloning. Hot cloning has been chosen to be analyzed because is the most common way of virtualizing servers used in production. Hot cloning uses volume based cloning therefore our attention will be mostly on this type of cloning. II. VOLUME-BASED CLONING Keywords— P2V,optimizing virtualization,VMware converter,hot cloning I. INTRODUCTION VMware vCenter Converter Standalone [1] is a product to convert virtual and physical machines to VMware virtual machines. When we convert a source machine, Converter Standalone uses cloning and system reconfiguration steps to create the destination virtual machine so that it works successfully in vCenter Server. The migration process does not delete or modify the source, therefore we can continue to use the original source machine after the conversion is completed. VMware Converter Standalone supports hot cloning, disk-based cloning, full or linked cloning. Hot cloning, also called live cloning permits converting the source machine while it is running its operating system [2]. Hot cloning permits cloning of the system without shutting it down. Because processes continue to run on the source machine during Volume-based cloning at the block level - performed when you choose to preserve the size of the source volume or when you specify a larger volume size for During volume-based cloning, volumes from the source machine are copied to the destination system. During volume-based cloning, all volumes of the destination virtual machine, except LVM2 logical volumes, are converted to basic volumes, regardless of their type in the corresponding source volume. LVM2 logical volumes can be maintained as logical volumes during conversion. Volume-based cloning can be performed at the file level or block level, depending on the destination volume size that we select.[1] Volume-based cloning at the file level – is performed when we select a size smaller than the original volume for NTFS volumes or you choose to resize a FAT volume. For FAT, FAT32, NTFS, ext2, ext3, ext4, XFS, and ReiserFS file systems, Converter Standalone preserves the file system type during a volume-based cloning at the file level. Dynamic source disks are read but not preserved during volume-based conversions. Dynamic disks will become basic volumes on the destination virtual machine. Volumebased cloning of dynamic source disks at the file level is available only for Windows. [1] NTFS source volumes. Volume-based cloning at the block level is supported only for Windows. [1] III. OPTIMIZING VOLUME-BASED CLONING For analyzing the best way of optimization for this type of cloning we will use the following test environment: We have a data center with only real servers and a blade server (E.g. IBM Blade center H22) that allows us virtualizing all existing servers. The blade server comes bundled with VMware vCenter Server v 5.1 We will convert 10 real systems with different operating systems to virtual environment in shortest time possible. Time limit is imposed also by environment: all real servers plus blade server cannot be sustained in good conditions by our air conditioning and power consumption is close to the limit of the electrical circuit [3]. Converting a single server could take from less than an hour to several days. If we multiply this process by 10 then we might need about a week just for conversion. Virtual servers will have also to be tested in the new environment to make sure that all processes are running properly. In a production environment these aspects are important. The destination VMware vCenter server load will increase after the virtualization. Before proceeding at adding a new virtual host to it we should consider the performance impact on it [5][6]. For first experiment, we will start the virtualization process using VMware Converter Standalone version 6 default settings. Default settings include:  SSL encryption for data transfer is enabled  Data connections per task set to 1 (found inside Administration menu)  Copy-type: Volume based  Parameters for destination virtual machine are not modified at all As we can see on Table 1, conversion times are affected mainly by HDD usage for Linux systems and by HDD capacity for Windows systems. By default Linux systems conversion is done using Volume-based cloning at file level and for Windows systems conversion is done using Volume-based cloning at the block level. Volumebased cloning at the file level is also possible for Windows systems but not recommended. Reducing volume size for file system volumes might render the destination machine unbootable. The conversion speed for file level cloning is slower than block level. On Table 1, HDD type refers to interface used by hard drive. Number of volumes, CPU or RAM didn’t affect too much the conversion time. TABLE 1. Volume based cloning – SSL enabled No Operating system No of volumes CPU RAM (GB) HDD Usage (GB) HDD capacity (GB) HDD type 1 Fedora Linux 13, x64 9 6 317 534 2 Fedora Linux 15, x32 8 2 97 3 Fedora Linux 19, x32 8 4 4 Fedora Linux 20,X64 8 5 Red Hat Linux Server 6.0, x64 4 6 Windows Server 2008, x64 Windows Server 2012, x64 Windows Server 2012, x64 Windows Server 2012, x64 Windows 7 Professional, x64 3 Intel Xeon L5335, 2.00GHz Intel Xeon X5675 , 3.07GHz Intel Xeon X3450, 2.67GHz Intel Xeon E31225,3.20GH z Intel Xeon X3450, 2.67GHz Intel Xeon L5335, 2.00GHz Intel Xeon X4860, 2.27GHz Intel Xeon X5675, 3.07Ghz Intel Xeon X5675, 3.07Ghz Intel I7–2600, 3.4 GHz 7 8 9 10 1 1 1 1 Conversion time (min) SCSI Average Transfer speed (MB/s) 29.4 250 SATA 32.5 51 121 500 SATA 34.4 60 16 750 2048 SATA 52.5 244 4 75 160 SATA 44.1 29 8 350 438 SCSI 32.8 182 8 111 250 SATA 47.4 40 8 142 250 SATA 49.5 49 8 119 250 SATA 47.2 43 16 175 250 SATA 49.8 60 184 A. Disable SSL data transfer encryption. Starting with VMware vCenter Converter Standalone 5.x, the converter worker encrypts the data stream using SSL. Encrypting the traffic increases the security, but it may decrease the performance. For disabling SSL encryption we have to edit configuration file converter-worker.xml [7] By default, this file is located at:  Windows 7/Server 2008 and C:\ProgramData\VMware\VMware Converter Standalone Later – vCenter  Windows Vista, XP and 2003 Server – %ALLUSERSPROFILE%\VMware\VMware vCenter Converter Standalone  Edit converter-worker.xml file using a text editor.  Inside the <nfc> tag locate, locate the line <useSsl>true</useSsl>  Replace the value to true with false. System No refers to system number column No found on any of mentioned tables. For this experiment the overall average conversion time increase was of 31.8 %. As we can see this is a major increase that we should take in consideration. If conversion speed is a priority and connection between real system and virtual system is secured enough to permit unsecured connections then virtualization should be done without having SSL encryption for data transfers. The most common configuration where source system and virtualization server are on the same room gives all premises for data transfer isolation and consequently SSL encryption for data transfers can be safely disabled. Convers i on time (min) To complete the conversion process faster we should consider the following:  On local machine, restart service VMware vCenter Converter Standalone Worker With the purpose of analyzing the improvement of conversion performance we have disabled SSL encryption of our VMware Converter application. The used operating system was Windows 7, x64 with 1 Gb/s network connection with source systems respectively our blade server. We have used as source the servers mentioned on Table 1. The conversion times without SSL encryption enabled are presented on Table 2. As we can see on Table 2, the column Conversion time (min) for each system has been reduced compared with previous test where SSL encryption was enabled. Consequently the Average transfer speed has been increased. A graphical representation of these results is presented on Fig. 1. Conversion Time (SSL) and Conversion Time (Non SSL) are the Conversion time (min) column from Table 1 respectively Table 2. 1 2 3 4 5 6 7 8 9 10 Fedora Linux 13, x64 Fedora Linux 15, x32 Fedora Linux 19, x32 Fedora Linux 20,X64 Red Hat Linux 6.0, x64 Windows Server 2008, x64 Windows Server 2012, x64 Windows Server 2012, x64 Windows Server 2012, x64 Windows 7 Professional, x64 100 0 1 2 3 4 5 6 7 8 9 10 Sys tem No Conversion time (non SSL) Fig. 1. VMware converter - SSL vs non SSL data connection B. Increase the value in the Number of Data Connections per Task option in Converter settings If we are converting the source with multiple disks and volumes, we can decrease the conversion time by cloning multiple disks and volumes concurrently. In the Converter GUI main menu, select Administration > Data Connections per Task to modify the settings. Cloning too many volumes from the same source disk in parallel can increase disk access times and consequently increase conversion time.[7] For this experiment we have considered a system with OS Ubuntu 14.04, x64 with a custom volume structure. CPU Intel Xeon x5675, 8 GB RAM, 250 GB SATA HDD with 11 volumes as follows:  8 custom volumes of 20 GB each with 10 GB usage  Root volume (/) - capacity of 20 GB – 10 GB TABLE 2. Volume based cloning - SSL disabled Operating system 200 Conversion time (SSL)  Save and close the file. No 300 HDD Usage (GB) 317 97 121 750 75 350 111 142 119 175 Average Transfer speed (MB/s) 38.9 40.4 45.9 70.7 55.7 44.2 61.1 63.8 63.5 69.5 Conversion time (min) % increase 139 41 45 181 23 135 31 38 32 43 32.4 24.4 33.3 34.8 26.1 34.8 29.0 28.9 34.4 39.5  Boot volume (/boot) – capacity of 20 GB – 10 Gb usage  Swap volume (/swap) – capacity 20 GB - no usage at the moment of test We have chosen this configuration to have a equilibrate test for multiple connections per task. Volume cloning was used for this tests because is the most common one. Custom volumes are having the same content. We have chosen files with size between several KB until 1 GB to cover the widest file sizes range. Root volume contains Ubuntu file system which initially was about 5 GB. This partition has been filled up with files from one of the custom volumes until we reached 10 GB usage. The same procedure was applied to boot volume. Initial usage of volume was 38 MB. If each volume size, usage and content would be completely different the results for this test could be affected. The total HDD usage for this system is 100 GB. We have considered this usage as optimum for testing purposes. Two tests was performed. One test with SSL encryption for data connection enabled and one test with SSL encryption disabled. On Table 3 from below are shown the conversion times for each test correlated to number of data connections. TABLE 3. Data connections per task Data connections per task Conversion time – SSL encryption enabled (min) 1 2 3 4 5 6 7 8 9 10 37.3 32.4 37.5 38 38.5 39 39.6 42 43 45.5 Conversion time – SSL encryption disabled (min) 27 24 28 28.4 29 29.5 30 31 32 35 The results achieved on Table 3 are represented graphically on from Fig.2. On the mentioned figure we can see that the best conversion time is achieved when we are using two data connections per task. When using more than two simultaneous data connections per task, the conversion time it is increasing continously, not making it a viable option for optimizing the coversion speed. Comversion time (min) usage 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 Data connections per task Conversion time (SSL) Conversion time (non SSL) Fig 2. Data connections per task – test results C. Use the adequate type of cloning In some cases, file level cloning may perform better than block level cloning. A source volume with a large amount of free space will be copied quicker using file cloning than block-level cloning. Using file level cloning to convert a source machine containing a large amount of small files can take a long time to copy. For this situation:  File level cloning is slower than Block level cloning.  Block level cloning is used when the destination volume size is maintained or increased.  File level cloning occurs when the destination volume is reduced in size during the conversion setup process. D. Clean the unnecessary files from the source server [7] Reducing the amount of source data will consequently reduce the conversion time. File level cloning will benefit the most from this solution. Block level cloning speed will not be improved because this type of cloning copies all data blocks, including free space. E. During conversion, we must ensure that network connection between the source and the destination is optimized (full duplex connections operating at full speed) [7]. Conversions using higher speed network links complete faster than conversions using slower links. F. Consider stopping services and applications on the source machine that are not critical for server functionality. [7] This will minimize modifications and consequently disk activity that occur in the source system while the conversion is in progress. Additionally, this reduces the amount of data that must be transferred during the conversion process. Example of this type of applications are: real time antivirus software, applications or databases that are not necessary during the conversion, monitoring software. G. Verify the physical RAID status in the source server. [7] A degraded RAID may significantly increase disk access times and as consequence increase conversion time. H. During the conversion process, do not run any I/O intensive activities in the source machine. [7] Anti-virus full file system scans, backups, disk defragmentation, and file system integrity checks could affect the conversion process. We must perform these activities in advance and not during the conversion process. I. Install the VMware converter software directly onto the source operating system using local Administrator account, if possible, otherwise, perform the remote conversion.[8] J. Leave the default number of virtual network cards (NICs) unchanged. [8] The number of virtual NICs can be changed after conversion has completed. K. Deselect the option to install VMware Tools.[8] L. Deselect the option to perform Customization on the virtual machine. [8] VMware doesn’t recommend virtualization of Windows domain controllers [9]. Because during the conversion, source system performs a lot important data changes the resulted system will not work properly. Domain controllers are also very sensitive to hardware changes. When a physical server is virtualized, the hardware presented to the operating system may be very different. It is possible in this case to have a virtualized domain controller and an identical physical domain controller running simultaneously, which may result in unpredictable replication issues across Active Directory. Windows Server 2012 is the only operating system that supports officially virtualization through VMware.[10] IV. CONCLUSIONS Virtualization is a must for nowadays datacenters. A key benefit of virtualization is the ability to consolidate and contain the number of servers needed, which, in turn, allows our organization to run multiple application and operating system workloads on one server. So far we have analyzed the best ways of optimization for volumebased cloning at file level respectively at volume-based cloning at block level. Each situation encountered along the way was analyzed and solutions were provided. Future research will concentrate on optimizing the other types of cloning supported by VMware Converter standalone: disk based respectively linked cloning. Disk based cloning can be a good solution for situations where hot cloning procedure fails. Linked cloning is useful when we need to share the same data between several virtual servers. ACKNOWLEDGMENTS This paper has been prepared with the financial support of the project “Quality European Doctorate EURODOC”, Contract no. POSDRU/187/1.5/S/155450, project co-financed by the European Social Fund through the Sectoral Operational Programme “Human Resources Development” 2007-2013. REFERENCES [1] VMware vCenter Converter Standalone 6.0 User's Guide, https://www.vmware.com/pdf/convsa_60_guide.pdf [2] P2V Migration with Post-Copy Hot Cloning for Service Downtime Reduction, Asai, H., Cloud and Green Computing (CGC), 2013 Third International Conference on, Year 2013, Pages: 1 - 8, DOI: 10.1109/CGC.2013.9, IEEE Conference Publications [3] NICE: Network-aware VM Consolidation scheme for Energy Conservation in Data Centers, Bo Cao; Xiaofeng Gao; Guihai Chen; Yaohui Jin, Parallel and Distributed Systems (ICPADS), 2014 20th IEEE International Conference on year: 2014, Pages: 166 - 173, DOI: 10.1109/PADSW.2014.7097805, IEEE Conference Publications [4] VMware vSphere Documentation, https://www.vmware.com/support/pubs/vsphere-esxi-vcenterserver-pubs.html [5] Live Migration Impact on Virtual Datacenter Performance: Vmware vMotion Based Study, Elsaid, M.E.; Meinel, C., Future Internet of Things and Cloud (FiCloud), 2014 International Conference on Year: 2014, Pages: 216 - 221, DOI: 10.1109/FiCloud.2014.42, IEEE Conference Publications [6] A Model of Storage I/O Performance Interference in Virtualized Systems, Casale, G.; Kraft, S.; Krishnamurthy, D., Distributed Computing Systems Workshops (ICDCSW), 2011 31st International Conference on year: 2011, Pages: 34 - 39, DOI: 10.1109/ICDCSW.2011.46, Cited by: Papers (6), IEEE Conference Publications [7] Increasing the speed of conversion when converting a physical or virtual machine using VMware Converter (2071014), http://kb.vmware.com/kb/2071014 [8] Best practices for using and troubleshooting VMware Converter (1004588), http://kb.vmware.com/kb/1004588 [9] Virtualizing existing domain controllers in VMware vCenter Converter (1006996), http://kb.vmware.com/kb/1006996 [10] Introduction to Active Directory Domain Services (AD DS) Virtualization (Level 100), http://technet.microsoft.com/enus/library/hh831734.aspx