XS Japan 2008 Ganeti English
•
1 like•695 views
This document summarizes a presentation given at the Xen Summit 2008 in Tokyo about challenges in managing large virtualized environments. The presentation discussed scaling a machine pool from 10 to 1,000 physical machines and how different challenges arise at each level, including hardware compatibility and automation. It also covered different types of virtual machines for servers, desktops, and labs and how to integrate them. Finally, it provided an overview of how Google uses Ganeti to manage its virtualized infrastructure by fully automating resource management across a large cluster of machines with varying hardware over time.
Report
Share
XS Japan 2008 Ganeti English
- 1. Xen Summit 2008 Tokyo Virtualization deployments A discussion of various challenges in virtualized environments Iustin Pop Ganeti team Google Switzerland 1 Copyright by Google Inc
- 2. Overview •Introduction •Abstract model •Scaling the machine pool size Up to 10 Up to 100 Up to 1,000 •VM types •Hardware evolution •Ganeti in Google 2 Copyright by Google Inc
- 3. Introduction Use cases of virtualization: •traditional: improved resource utilization consolidate lowusage machines •but the technology has matured: focus moves to higher layers (management) large numbers of both physical and virtual machine more uses (VM types) Disclaimer: •the presentation content is not representative of Google's usage of virtualization •the presentation solely refers to the use of virtualization in Google for internal, corporate purposes and not external services or products (e.g. www.google.com) 3 Copyright by Google Inc
- 4. Abstract model app app app app app er monitoring lay machine lifecycle controls n access tio ica ppl a app er lay n monitoring tio cluster machine ra lifecycle controls eg access int resource mgmt. machine mt resource mgmt. e mg urc reso hypervisor resource mgmt. base OS base OS hardware layer hardware hardware multiple app single-app or virtualized 4 Copyright by Google Inc
- 5. Scaling the number of machines The machine pool size influences the ROI for different features •at lower sizes, efficiency at lower levels is most important •growing the number of machines increases the importance of the management layer •HW failure rate importance varies •automation cost is moreorless constant, but benefits vary greatly 5 Copyright by Google Inc
- 6. Up to 10 physical machines Main characteristics: •Hypervisor and HW efficiency is paramount •Small number of machines translates into: small customer base low number of HW failures reduced benefit of automation greater chance of same HW profile Challenges: •application compatibility 6 Copyright by Google Inc
- 7. Up to 100 physical machines Similar to nonvirtualized environments: •component failure is rare, but part of normal life •automation benefits start to show (deployment, configuration, etc) Specific to virtual environments: •diverse configurations and machine mobility mean VMs will be shifted around and their HW profile can change dynamically •cost savings are split between resource utilization and operational gains Challenges: •accommodating various customer types •integration of VM and nonVM environments 7 Copyright by Google Inc
- 8. Up to 1,000 physical machines Number of machines affect cost profile: •HW costs are linear, but operations costs no longer •Automation and standardization across all layers have big benefits the hypervisor and HW layers can be deeply abstracted by the management tools automation of all procedures is paramount to keeping the VM environment healthy Challenges: •multiple customers, same management toolset •software upgrades for physical machines •dealing with multiple HW generations 8 Copyright by Google Inc
- 9. VM types Deployments will have multiple VM types: •server/desktop/lab/etc. •central administration versus enduser administration •integration of all these into the same machine pool 9 Copyright by Google Inc
- 10. VM types server •stable, longlife •monitoring is important •resource usage has smoother variation •usually continuous operation 10 Copyright by Google Inc
- 11. VM types desktop •less stable life •bursty usage •'business hours' type of operation •GUI/user friendly interface to VMspecific operations very important •monitoring integration good for debugging, but less for the big mass of end users 11 Copyright by Google Inc
- 12. VM types lab •short lived •enduser provisioning out of own resource pool •monitoring less useful than quick provisioning •snapshot, rollback and similar features important 12 Copyright by Google Inc
- 13. Keeping pace with hardware Commodity hardware: •rapidly changing specifications: cores memory disk size •and not so rapid: network/disk bandwidth singleCPU speed This results in asymmetric growth: •resource allocation needs to handle this •cluster architecture changes over time 13 Copyright by Google Inc
- 14. Ganeti @ Google Design goals and principles •sits between hardware layer and integration layer •fully automate node and clusterlevel resource management •needs base OS and hypervisor •does not provide monitoring, access controls or global provisioning (across clusters) •principles: not dependent on specific hardware (e.g. external shared storage) scales (almost) linearly with the number of systems 14 Copyright by Google Inc