VMworld 2016: Virtual Volumes Technical Deep Dive
•Download as PPTX, PDF•
3 likes•1,207 views
Virtual Volumes provide a more efficient operational model for external storage management in vSphere. They integrate storage capabilities directly into virtual machines at the individual disk level through Storage Policy-Based Management. This simplifies operations by removing the need for static LUN/volume provisioning and allows storage services to be applied non-disruptively on a per-virtual machine basis according to policies. A key component is the VASA Provider, which is used to publish an array's storage capabilities and manage the creation of VM-level objects called Virtual Volumes on behalf of vSphere.
Report
Share
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (15)
Similar to VMworld 2016: Virtual Volumes Technical Deep Dive
Similar to VMworld 2016: Virtual Volumes Technical Deep Dive (20)
More from VMworld
More from VMworld (7)
Recently uploaded
Recently uploaded (20)
VMworld 2016: Virtual Volumes Technical Deep Dive
- 1. Virtual Volumes Technical Deep Dive Patrick Dirks, VMware, Inc Pete Flecha, VMware, Inc STO7645 #STO7645
- 2. • This presentation may contain product features that are currently under development. • This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. • Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. • Technical feasibility and market demand will affect final delivery. • Pricing and packaging for any new technologies or features discussed or presented have not been determined. Disclaimer CONFIDENTIAL 2
- 3. 3 Traditional Storage is Difficult to Align to Application Specific Needs CONFIDENTIAL
- 4. Customers Face Several Challenges with Storage Today 4 Fragmentation of roles Specific management tools Specialized skillsets Static classes of service Lack of fine grained control Frequent data migrations Time consuming processes Lack of automation Slow reaction to request Rigid Infrastructure Complex ProcessesSiloed Management CONFIDENTIAL
- 5. Key Issues With Traditional Storage Management 5 Goal: Deliver Storage Requirements To Workloads Consistently And With Agility Capacity (per-Disk) Capacity Performance Capabilities Resiliency (per-LUN) Capabilities applied at LUN/Volume level2.No visibility to storage capabilities during provisioning1. Vendor specific configuration required in advance of workload demand3. CONFIDENTIAL
- 6. Legacy Storage Model Introduces Lots of Complexity 6 Datastores Datastore Clusters Reservations Finance Gold Finance Silver Finance Bronze Reservation Policy HR Gold HR Silver HR Bronze Business Groups Finance HR vSphereVRA CONFIDENTIAL
- 7. Align Application Requirements with Infrastructure CONFIDENTIAL 7 Goal: Deliver Storage Requirements To Workloads Consistently And With Agility Capabilities applied at Virtual Disk level via SPBM Storage Policies 2.Storage capabilities integrated via Storage Policy-Based Management1. Consumed on demand via VVols/VSAN/VAIO3. Capacity Performance Capabilities Resiliency (per-VM/Disk)
- 8. Storage Policy-Based Model Simplifies Operations Significantly 8 Finance Gold Finance Silver Finance Bronze HR Gold HR Silver HR Bronze Finance HR Datastores Datastore Clusters Reservations Reservation Policy Business Groups vSphereVRA CONFIDENTIAL
- 9. Agenda 9 1 What are VVols? 2 Architecture 3 VVols in Operation 4 Questions and Answers CONFIDENTIAL
- 10. vSphere Virtual Volumes A More Efficient Operational Model for External Storage
- 11. VMware vSphere Virtual Volumes 11 Integration Framework for VM-Aware Storage Virtual Volumes Overview • Virtual disks are natively represented on arrays • Enables VM granular storage operations using array-based data services • Extends vSphere Storage Policy-Based Management to the storage ecosystem • Supports existing storage I/O protocols (FC, iSCSI, NFS) • Based on T10 industry standards • Industry-wide initiative supported by major storage vendors • Included with vSphere CONFIDENTIAL
- 12. Virtual Volumes Introduces a New Architecture 12 • Arrays logically partitioned into Storage Containers (i.e. Separate HR, Finance) • Bi-directional out of band communication with VASA Provider • VASA Provider specific to vendor • No Filesystem • VM-granular storage policy-based management via SPBM and VASA • Data Services offloaded to the Array • Protocol Endpoint (PE) directs IO from VM to Virtual Volume • Non disruptive – data path follows standard T10/NFS industry standards VVol-enabled array PE VP Data Path NFS, iSCSI, FC Control Path VASA APIs Protocol Endpoint VASA Provider vSphere Storage Policy-Based Management IO Device Finance HR CONFIDENTIAL
- 13. vSphere Virtual Volumes vSphere Web Client View vvol CONFIG DATA SWAP MEM • Virtual Volumes – Virtual machine objects stored natively on the array. – No Filesystem on-disk formatting required • There are five different types of recognized Virtual Volumes: – CONFIG – vmx, logs, nvram, log files, etc – DATA – VMDKs – MEM – Snapshots – SWAP – Swap files – Other – vSphere solution specific type CONFIDENTIAL 13
- 14. Data Plane vSphere Virtual Volumes
- 15. Storage Container Storage Containers • Logical storage constructs for allocating array resources • Typically defined and setup by storage administrators on the array in order to define: – Storage capacity allocations and restrictions – Capabilities to be presented • Minimum one storage container per array • Maximum depends on the array vSphere Virtual Volumes SAN / NAS Storage Containers HR Finance CONFIDENTIAL 15
- 16. Differences Between Storage Containers and LUNs CONFIDENTIAL 16 • Fixed size mandates greater number of LUNs • Formatted with a fixed-size file system on every LUN • Changing LUN size requires reformatting • Provisioned using file system commands • Can only apply homogeneous capability on all VMs (VMDKs) provisioned in that LUN. • Size limit selected by Storage administrator • Size can be adjusted up/down as needed on the fly • Max number of Storage Containers depend on the array ability and capacity • Can support different capabilities for different VMs or VVols in that Storage Container using SPBM Storage Containers LUNs
- 17. Discovery Procedures – Storage Container CONFIDENTIAL 17 Virtual Datastore Virtual Volumes HR Finance vCenter VASA Provider Storage Container Discovery Process • Storage admin sets up Storage Container with desired capacity limit and selection of capabilities • VASA Provider presents Storage Container and container’s capabilities to vCenter • vSphere admin creates “Virtual Datastores” on each host to grant access to Storage Container • Any new VMs that are created will subsequently be provisioned in the Virtual Datastore depending on match with storage policies
- 18. Protocol Endpoints Protocol Endpoints • Access points that enables communication between ESXi hosts and storage array systems. – Part of the physical storage fabric – Created by Storage administrators Scope of Protocol Endpoints • Compatible with all SAN and NAS Protocols: - iSCSI - NFS v3 - FC - FCoE • A Storage Container’s PEs must all be either SAN or NAS • Existing multi-path policies and NFS topology requirements can be applied to the PE SAN / NAS Virtual Datastore Data Path Protocol Endpoint vSphere Virtual Volumes Storage Container PEPE Protocol Endpoint CONFIDENTIAL 18
- 19. Protocol Endpoints • Today, there are different types of logical management constructs to store VMDKs/objects: – NFS Mount Points – IP or block based datastores • Datastores serve two purposes today: – Endpoints – receive SCSI or NFS reads, write commands – Storage Container – for large number of VMs metadata and data files • Differences between Protocol Endpoints and Datastores: – PEs no longer stores VMDKs but it only becomes the access point. – Now you wont need as many datastores or mount point as before • Certain offloading operation will be done via VASA and other will be done using the standard protocol commands vSphere storage fabric PE Protocol Endpoint SCSI: proxy LUN NFS: mount-point storage system 1 VVol (storage container) Per VMDK One entity on the fabric CONFIDENTIAL 19
- 20. Discovery Procedures – Protocol Endpoint Protocol Endpoint discovery process • SCSI PEs are discovered by hosts during an ESX rescan as usual • NFS PEs are configured using IP addresses or file paths and stored on hosts • Hosts inform array’s VP about PEs found and accessible SAN / NAS Virtual Datastore Data Path Protocol Endpoint vSphere Virtual Volumes Storage Container PE PE Protocol Endpoint CONFIDENTIAL 20
- 21. Management Plane vSphere Virtual Volumes
- 22. Storage Policy-Based Management – App Centric Automation 22 Overview • Intelligent placement • Fine control of services at VM or even individual disk level • Automation at scale through policy • Need new services for VM? • Change current policy on-the-fly • Attach new policy on-the-fly Virtual Machine Storage policy Reserve Capacity 40GB Deduplication On Disk Type SSD Disk Encryption Off Storage Policy-Based Management vSphere Virtual SAN Virtual Volumes Virtual Datastore CONFIDENTIAL
- 23. VASA Provider (VP) CONFIDENTIAL 23 SAN / NAS Virtual Datastore Data Path vSphere Virtual Volumes Storage Container VASA Provider (VASA) Control Path Control Path PE PE • Software component developed by Storage Array Vendors • ESX and vCenter Server connect to VASA Provider • Provides Storage awareness services • Single VASA Provider can manage multiple arrays • Supports VASA API Suite • VASA Provider can be implemented within the array’s management server or firmware or as a VM • Responsible for creating and managing Virtual Volumes
- 24. Storage Capabilities and VM Storage Policies SPBM object manager virtual disk Datastore Profile VM Storage Policy vSphere VM Storage Policy Management Framework Storage Capabilities for Storage Array Access Capacity Published Capabilities Compression Replication Deduplication QoS Virtual Datastore • Storage Capabilities – are array based features and data services specifications that capture storage requirements that can be satisfied by a storage arrays advertised as capabilities. • Storage capabilities define what an array can offer to storage containers as opposed to what the VM requires. • Arrays Storage Capabilities are advertises to vSphere through the Vendor Provider and VASA APIs • In vSphere Storage Capabilities are consumed via VM Storage Policy constructs. • VM Storage Policies is a component of the vSphere Storage Policy-based management framework (SPBM) CONFIDENTIAL 24
- 25. Array Capabilities and Storage Policies Storage Capabilities to Storage Policies • Storage Admin creates Storage Containers, selecting • size limit • capabilities • VI Admin registers VP • VP presents Storage Containers and associated capabilities to vSphere • VI Admin authors VM Storage Polices based on presented capabilities to match expected VM needs • VI Admin assigns policies as VMs are provisioned • Storage policies determine selection of datastore Storage Policy Capacity Availability Performance Data Protection Security Published Capabilities Compression Replication Deduplication Encryption vSphere Storage Policy-Based Mgmt. Virtual Volumes VASA ProviderPE PE CONFIDENTIAL 25
- 26. Compliance Made Easy With Automated Policy Monitoring 26CONFIDENTIAL
- 27. Management Workflow • What do the Admins see? • How are the storage containers setup? • What does the vSphere Admins see? • Why are we still creating datastores in this new model? CONFIDENTIAL 27 Storage policies vSphere Web Client Storage Management UI Datastore Storage Container Storage Capabilities virtual volumes virtual machines
- 28. VVols in Operation Provisioning and Lifecycle Workflows
- 29. Storage Container vvol DATA vvol CONF vvol SWAP vvol DATA vvol CONF vvol SWAP Provisioning Workflow storage arrays PE vSphere Admin 1. Create Virtual Machines 2. Assign a VM Storage Policy 3. Choose a suitable Datastore Under the Covers Provisioning operations are translated into VASA API calls in order to create the individual virtual volumes, passing storage policies to array Under the Covers Provisioning operations are offloaded to the array for the creation of virtual volumes on the storage container that are stored based on capabilities defined in the VM Storage Policies offloadtoarray Virtual Datastore vSphere Virtual Volumes PE PE CONFIDENTIAL 29
- 30. Binding Operations create bind unbinddelete VM Creation VM power-on open (2) I/O read (2) I/O write (2) VM power-off close (2) VM destroy VP rebalance REBIND I/O Virtual Volume Lifecycle • Bindings are data path coordinating mechanisms that occurs between VASA providers and ESXi hosts for accessing virtual volume. • Different Binding Mechanism: • Binding – allows array create I/O channels for a virtual volume • Unbind – destroys the I/O channel for a virtual volume to a given ESXi host • Rebind – provides the ability to change the I/O path (i.e. choice of PE) for a given virtual volumes run time using array-generated events. CONFIDENTIAL 30
- 31. Snapshots • Snapshots are a fixed point in time image of a Virtual Volume with a different ID from the original. • Snapshots may be read-only or read/write • Virtual Volumes snapshots are useful in the contexts of creating: – a quiesced copy for backup or archival purposes, creating a test and rollback environment for applications. • Two type of snapshots supported: – Managed Snapshot – Created and managed by vSphere. • A maximum of 32 vSphere managed snapshot are supported for linked clones of an individual VM – Unmanaged Snapshot – Created and managed by the storage array. Managed Snapshot - vSphere Unmanaged Snapshot - Array CONFIDENTIAL 31
- 32. Snapshots: Files vs VVols 32 main.vmdk VMFS VVol VVol ID 42 main.vmdk snap.vmdk VVol ID 42 Flat file snap.vmdk Redo log Redo log snap.vmdk VVol ID 86 CONFIDENTIAL
- 33. VVol ID 861 Reverting Snapshots 33 main.vmdk VVol ID 42 main.vmdk snap.vmdk VVol ID 42 Flat file VVol ID 86 backout VVol ID 243 VMFS VVol CONFIDENTIAL
- 35. Integrate Data Services Simplify Management Support Industry Leading Virtualization Platform Replication Encryption Device advancements Usability Operate at scale Virtual disk level control Scalability Performance Integration Future Themes for Storage Management 35CONFIDENTIAL
- 36. VASA Replication Model 36 Overview • Replication is modeled as relationship (“Replication Group”) between “Fault Domains” • Topology Discovery (Fault Domains, Replication Groups) through VASA APIs • Replication operations defined on “Replication Groups” such as – Synchronize – Failover – TestFailover – Reverse Replication Future Considerations New York Boston Palo Alto HR Replication Group Marketing Replication Group Finance Replication Groups CONFIDENTIAL
- 37. What Do Customers Want? 37 Application Compute Storage Optimize Infrastructure Simplify OperationsDeliver Agility CONFIDENTIAL
- 38. What Do Customers Like About Virtual Volumes 38 Application driven provisioning Rapid automation Faster VM operations Streamlined administration Policy driven offerings Flexible levels of control Better storage utilization Avoid over provisioning VM-aware space reclamation Application Compute Storage Optimize Infrastructure Simplify OperationsDeliver Agility CONFIDENTIAL
- 39. Certified Partners with Solutions vSphere Virtual Volumes is an Industry-wide Initiative 39 All Industry Leaders >30 Partners Involved >40 VVol-Enabled Arrays CONFIDENTIAL
- 40. Come Join the Discussion STO10719-GD Group Discussion: Meet the Virtual Volumes Product and Engineering Team Today at 1:00 PM Virtual Volumes CONFIDENTIAL 40
- 41. Come Join the Discussion Virtual Volumes STO8619 Transitioning to VVols: Partner Panel Wednesday at 4:00 PM CONFIDENTIAL 41
- 42. Subscribe to the vSpeaking Podcast vSpeakingPodcast.com CONFIDENTIAL 42
- 43. Questions