Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

VMworld Europe 2014: Storage DRS - Deep Dive and Best Practices

•
•

This document discusses new features in VMware vSphere 6.0 related to storage management and optimization. It introduces Storage DRS which helps balance storage resource utilization across datastores and hosts. New features like IO reservations allow minimum guaranteed performance levels for VMs. Storage DRS also integrates with storage array technologies like thin provisioning, deduplication, auto-tiering, and replication to improve storage efficiency and availability. The document provides best practices for deploying Storage DRS to maximize its benefits.

1 of 36
Download to read offline
VMworld Europe 2014: Storage DRS - Deep Dive and Best Practices
Disclaimer • This presentation may contain product features that are currently under development. • This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. • Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. • Technical feasibility and market demand will affect final delivery. • Pricing and packaging for any new technologies or features discussed or presented have not been determined. CONFIDENTIAL 2
Storage DRS Deployment: Thin Provisioning Deduplication Auto-Tiering Outline Storage IO Control Capabilities New Control: IO Reservations Storage DRS Storage IO Control New ESX IO Scheduler New Control: IO Reservations Storage DRS Integration Array-Based ReplicationStorage DRS Integration vSphere Replication
vSphere 6.0 Beta • Features we are presenting here are available in beta • Perfect time to get your voice heard https://communities.vmware.com/community/vmtn/vsphere-beta 4
The Problem What you see Online store: Product Catalog Back-up (low priority) Shared Datastore Online Store: Order Processing What you want to see Shared Datastore Online store: Product Catalog Back-up (low priority) Online Store: Order Processing 5
Storage Performance Controls  Shares: Relative importance of VMs – IOPS will be allocated in this proportion  Limit: Maximum IOPS allowed per VM  Reservations: Minimum IOPS per VM IOPS: 1200 600 600 IOPS: 1200 400 400 400 Online store: Product Catalog Back-up (low priority) Online Store: Order Processing R: 100 L: 500 S: 20S: 100S: 100
Outline New ESX IO Scheduler New Control: IO Reservations
ESX 5.5 I/O Scheduler (mClock) • Allocation of A and B will be in 1000:3000 ratio subject to reservation and limit constraint. L: ∞ S: 3000 R: 100 L: 500 S: 1000 R: 200 Capacity (IOPS) VM A (IOPS) VM B (IOPS) 1000 250 750 500 200 300 200 133.33 66.66 A B 8
ESX 5.5 I/O Scheduler (mClock)  Compared to previous scheduler: – Supports Reservation, Shares, and Limits – Break large I/Os in 32KB Limit: 100 IOPS IO Size: 64KB 100 IOPS Limit: 100 IOPS IO Size: 64KB 50 IOPS ESX 4.1 ESX 5.5
ESX 5.5 I/O Scheduler (mClock)
Outline Storage IO Control Capabilities New Control: IO Reservations Storage IO Control New ESX IO Scheduler New Control: IO Reservations
Shared Storage S: 100 SIOC Local I/O Scheduler Local I/O Scheduler + Storage I/O Control S: 100 S: 100 S: 100 S: 100 S: 100 1200 IOPS600 IOPS 600 IOPS 1200 IOPS 600300300 400 400 400300 300 600
Storage IO Control  Control Congestion in shared datastore  Detect Congestion – SIOC monitors average IO latency for a datastore – Latency above a threshold indicates congestion  SIOC throttles IOs once congestion is detected – Control IOs issued per host – Based on VMs shares, reservations, and limits on each host – Throttling adjusted dynamically based on workload • Idleness • Bursty behavior
Congestion Threshold  Datastore overload  Congestion threshold value (ms): – Higher is better for overall throughput – Lower is better for latency  Changing default threshold:  Percentage or absolute value  Default: 90% of peak IOPs capacity Throughput(IOPS) Datastore Load No benefit beyond certain load Latency Datastore Load
SIOC and Reservations  SIOC needs to be aware of VMs reservations – The queue depth allocation of each host depends on the VM reservation  Backward compatible (ESX 5.5 or below)  Are reservations always satisfied? – IOPS too low due to low latency threshold – Background operations or errors in the array  In case reservations are not satisfied: – SIOC will notify Storage DRS for further action R: 500 SIOC R: 500 R: 500
Outline Storage IO Control Capabilities New Control: IO Reservations Storage DRS Storage IO Control Storage DRS: IO Reservations Functionality Overview
Storage DRS and IO Reservations • Monitor SIOC reservation enforcement – Migrate workload if reservations are not met • Balance reserved IOPS usage in cluster – Match datastore IOPS capacity to reservations • Reservations as VM placement constraints – Hard or soft constraints • Per-datastore reservable IOPS – Manual override allowed Datastore Cluster 17
 Ease of Storage Management  Initial Placement  Out of Space Avoidance  IO Load Balancing  Virtual Disk Affinity (Anti-Affinity)  Datastore Maintenance Mode  Add Datastore Storage DRS Functionality: Brief Overview Datastore Cluster Storage vMotion •••
Storage DRS Deployment: Thin Provisioning Deduplication Auto-Tiering Outline Storage DRS
Datastore Capacity Management Used Space : 700 GB Thin VMDK: 100 GBThick VMDK: 500 GB Datastore Free: 1000 GBDatastore Free: 500 GBDatastore Free: 400 GB Thin VMDK: 100 GB Datastore Free: 300 GB Storage DRS Configuration Option: PercentIdleMBInSpaceDemand, default = 25% Provisioned Space : 1500 GB Space Demand : 900 GB 20
Datastore: 2 TB Thin Provisioned Datastores Storage Array Datastore: 1 TB Bac Datastore: 1 TB Capacity : 4 TB Used Capacity : 4 TB Provisioned Capacity : 12 TB Allocated Capacity : 6 TB Virtual Disks Backing Storage Pool 21
How does Storage DRS handle space over-commit? • Hypervisor Level: Storage DRS manages space over-commit – Fine grain control for overcommit magnitude – Out of space avoidance using storage vmotion • Datastore Level: Storage arrays manage space over-commit – Storage DRS manages logical space usage in datastores – VASA integration (v1): Storage DRS handles space outage signal from backing pool – VASA integration (v2): Storage DRS controls space usage in backing pools • Best practice: can use either method, but not both 22
Deployment with Deduplication • Provides space efficiency • Dedupe pool can span across multiple datastores Dedupe Storage DRS uses free space in datastore VASA Integration (v2) : Storage DRS manages logical space while keeping virtual disks in the same dedupe pool ⤬Problem: Datastore appears to store more data than capacity! Total Virtual Disk Size: 4TB LUN Capacity: 1TB 23
Deployment with Auto-Tiered Arrays • Multiple storage tiers • VM data across tiers • Tier use changes with workload Capacity Tier Performance Tier Logical LUN of Auto-tier Array Storage IO Control  IO priority IO over-load remediation Automatic initial placement Space load balancing Rule enforcement Maintenance mode Latency IO Load Auto-tiered array SIOC Threshold SDRS Threshold 24
Fine Grain Controls All aspects of Storage DRS can be controlled to suit your environments
Storage DRS Deployment: Thin Provisioning Deduplication Auto-Tiering Outline Storage DRS Storage DRS Integration Array-Based Replication
vSphere SRM: Array-based Replication • Storage DRS identifies replicated datastores • All recommendations are in sync with replication policies: – Automated moves within the same consistency group – Manual moves for all VMs residing on replicated datastores • Accounting of replication overhead due to Storage vMotion 27
Outline Storage DRS Storage DRS Integration Array-Based ReplicationStorage DRS Integration vSphere Replication
vSphere Replication (VR) • Storage DRS discovers VR-replicas in datastores • Storage DRS understands space usage of replica disks • Storage coordinates moves with VR – Space balancing – Maintenance mode 29
Storage IO Control Best Practices  Avoid mixing vSphere LUNs and non-vSphere LUNs on the same physical storage – SIOC will detect this and raise an alarm  Configure host IO queue size with highest allowed value – Maximum flexibility for SIOC throttling  Keep congestion threshold conservatively high – Will improve overall utilization – Set lower if latency is more important than throughput
Datastore Cluster Best Practices  Similar datastore performance  May not be identical  Similar capabilities  Data management  Backup ✔Cluster1: Wide Perf Variation Cluster2: Similar Datastores 31
Datastore and Host Connectivity • Maximum possible host and datastore connectivity • Improves DRS and Storage DRS performance Partially Connected Datastore Cluster Fully Connected Datastore Cluster  More datastores in cluster  better space and I/O balance  Larger datastore size  better space balance DRS Cluster DRS Cluster 32
vSphere Storage Policy based Management Silver Disk Pool Gold Disk Pool Data store1 Data store2 Data store3 Data store4 Cluster-1 (Tier2 VMs) Cluster-2 (Tier1 VMs) Datastores with any storage profile Silver Disk Pool Gold Disk Pool Data store1 Data store2 Data store3 Data store4 Cluster-1 (Tier1 + Tier2 VMs) Datastores with identical storage profile 33
Summary • Easier Storage Management • Effective storage capacity and performance management – Support for IO reservations – Integration with storage environments – Integration with vSphere solutions • Many exciting features available in vSphere 6.0 Beta! 34 https://communities.vmware.com/community/vmtn/vsphere-beta
VMworld Europe 2014: Storage DRS - Deep Dive and Best Practices
VMworld Europe 2014: Storage DRS - Deep Dive and Best Practices

More Related Content

VMworld Europe 2014: Storage DRS - Deep Dive and Best Practices

  • 2. Disclaimer • This presentation may contain product features that are currently under development. • This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. • Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. • Technical feasibility and market demand will affect final delivery. • Pricing and packaging for any new technologies or features discussed or presented have not been determined. CONFIDENTIAL 2
  • 3. Storage DRS Deployment: Thin Provisioning Deduplication Auto-Tiering Outline Storage IO Control Capabilities New Control: IO Reservations Storage DRS Storage IO Control New ESX IO Scheduler New Control: IO Reservations Storage DRS Integration Array-Based ReplicationStorage DRS Integration vSphere Replication
  • 4. vSphere 6.0 Beta • Features we are presenting here are available in beta • Perfect time to get your voice heard https://communities.vmware.com/community/vmtn/vsphere-beta 4
  • 5. The Problem What you see Online store: Product Catalog Back-up (low priority) Shared Datastore Online Store: Order Processing What you want to see Shared Datastore Online store: Product Catalog Back-up (low priority) Online Store: Order Processing 5
  • 6. Storage Performance Controls  Shares: Relative importance of VMs – IOPS will be allocated in this proportion  Limit: Maximum IOPS allowed per VM  Reservations: Minimum IOPS per VM IOPS: 1200 600 600 IOPS: 1200 400 400 400 Online store: Product Catalog Back-up (low priority) Online Store: Order Processing R: 100 L: 500 S: 20S: 100S: 100
  • 7. Outline New ESX IO Scheduler New Control: IO Reservations
  • 8. ESX 5.5 I/O Scheduler (mClock) • Allocation of A and B will be in 1000:3000 ratio subject to reservation and limit constraint. L: ∞ S: 3000 R: 100 L: 500 S: 1000 R: 200 Capacity (IOPS) VM A (IOPS) VM B (IOPS) 1000 250 750 500 200 300 200 133.33 66.66 A B 8
  • 9. ESX 5.5 I/O Scheduler (mClock)  Compared to previous scheduler: – Supports Reservation, Shares, and Limits – Break large I/Os in 32KB Limit: 100 IOPS IO Size: 64KB 100 IOPS Limit: 100 IOPS IO Size: 64KB 50 IOPS ESX 4.1 ESX 5.5
  • 10. ESX 5.5 I/O Scheduler (mClock)
  • 11. Outline Storage IO Control Capabilities New Control: IO Reservations Storage IO Control New ESX IO Scheduler New Control: IO Reservations
  • 12. Shared Storage S: 100 SIOC Local I/O Scheduler Local I/O Scheduler + Storage I/O Control S: 100 S: 100 S: 100 S: 100 S: 100 1200 IOPS600 IOPS 600 IOPS 1200 IOPS 600300300 400 400 400300 300 600
  • 13. Storage IO Control  Control Congestion in shared datastore  Detect Congestion – SIOC monitors average IO latency for a datastore – Latency above a threshold indicates congestion  SIOC throttles IOs once congestion is detected – Control IOs issued per host – Based on VMs shares, reservations, and limits on each host – Throttling adjusted dynamically based on workload • Idleness • Bursty behavior
  • 14. Congestion Threshold  Datastore overload  Congestion threshold value (ms): – Higher is better for overall throughput – Lower is better for latency  Changing default threshold:  Percentage or absolute value  Default: 90% of peak IOPs capacity Throughput(IOPS) Datastore Load No benefit beyond certain load Latency Datastore Load
  • 15. SIOC and Reservations  SIOC needs to be aware of VMs reservations – The queue depth allocation of each host depends on the VM reservation  Backward compatible (ESX 5.5 or below)  Are reservations always satisfied? – IOPS too low due to low latency threshold – Background operations or errors in the array  In case reservations are not satisfied: – SIOC will notify Storage DRS for further action R: 500 SIOC R: 500 R: 500
  • 16. Outline Storage IO Control Capabilities New Control: IO Reservations Storage DRS Storage IO Control Storage DRS: IO Reservations Functionality Overview
  • 17. Storage DRS and IO Reservations • Monitor SIOC reservation enforcement – Migrate workload if reservations are not met • Balance reserved IOPS usage in cluster – Match datastore IOPS capacity to reservations • Reservations as VM placement constraints – Hard or soft constraints • Per-datastore reservable IOPS – Manual override allowed Datastore Cluster 17
  • 18.  Ease of Storage Management  Initial Placement  Out of Space Avoidance  IO Load Balancing  Virtual Disk Affinity (Anti-Affinity)  Datastore Maintenance Mode  Add Datastore Storage DRS Functionality: Brief Overview Datastore Cluster Storage vMotion •••
  • 19. Storage DRS Deployment: Thin Provisioning Deduplication Auto-Tiering Outline Storage DRS
  • 20. Datastore Capacity Management Used Space : 700 GB Thin VMDK: 100 GBThick VMDK: 500 GB Datastore Free: 1000 GBDatastore Free: 500 GBDatastore Free: 400 GB Thin VMDK: 100 GB Datastore Free: 300 GB Storage DRS Configuration Option: PercentIdleMBInSpaceDemand, default = 25% Provisioned Space : 1500 GB Space Demand : 900 GB 20
  • 21. Datastore: 2 TB Thin Provisioned Datastores Storage Array Datastore: 1 TB Bac Datastore: 1 TB Capacity : 4 TB Used Capacity : 4 TB Provisioned Capacity : 12 TB Allocated Capacity : 6 TB Virtual Disks Backing Storage Pool 21
  • 22. How does Storage DRS handle space over-commit? • Hypervisor Level: Storage DRS manages space over-commit – Fine grain control for overcommit magnitude – Out of space avoidance using storage vmotion • Datastore Level: Storage arrays manage space over-commit – Storage DRS manages logical space usage in datastores – VASA integration (v1): Storage DRS handles space outage signal from backing pool – VASA integration (v2): Storage DRS controls space usage in backing pools • Best practice: can use either method, but not both 22
  • 23. Deployment with Deduplication • Provides space efficiency • Dedupe pool can span across multiple datastores Dedupe Storage DRS uses free space in datastore VASA Integration (v2) : Storage DRS manages logical space while keeping virtual disks in the same dedupe pool ⤬Problem: Datastore appears to store more data than capacity! Total Virtual Disk Size: 4TB LUN Capacity: 1TB 23
  • 24. Deployment with Auto-Tiered Arrays • Multiple storage tiers • VM data across tiers • Tier use changes with workload Capacity Tier Performance Tier Logical LUN of Auto-tier Array Storage IO Control  IO priority IO over-load remediation Automatic initial placement Space load balancing Rule enforcement Maintenance mode Latency IO Load Auto-tiered array SIOC Threshold SDRS Threshold 24
  • 25. Fine Grain Controls All aspects of Storage DRS can be controlled to suit your environments
  • 26. Storage DRS Deployment: Thin Provisioning Deduplication Auto-Tiering Outline Storage DRS Storage DRS Integration Array-Based Replication
  • 27. vSphere SRM: Array-based Replication • Storage DRS identifies replicated datastores • All recommendations are in sync with replication policies: – Automated moves within the same consistency group – Manual moves for all VMs residing on replicated datastores • Accounting of replication overhead due to Storage vMotion 27
  • 28. Outline Storage DRS Storage DRS Integration Array-Based ReplicationStorage DRS Integration vSphere Replication
  • 29. vSphere Replication (VR) • Storage DRS discovers VR-replicas in datastores • Storage DRS understands space usage of replica disks • Storage coordinates moves with VR – Space balancing – Maintenance mode 29
  • 30. Storage IO Control Best Practices  Avoid mixing vSphere LUNs and non-vSphere LUNs on the same physical storage – SIOC will detect this and raise an alarm  Configure host IO queue size with highest allowed value – Maximum flexibility for SIOC throttling  Keep congestion threshold conservatively high – Will improve overall utilization – Set lower if latency is more important than throughput
  • 31. Datastore Cluster Best Practices  Similar datastore performance  May not be identical  Similar capabilities  Data management  Backup ✔Cluster1: Wide Perf Variation Cluster2: Similar Datastores 31
  • 32. Datastore and Host Connectivity • Maximum possible host and datastore connectivity • Improves DRS and Storage DRS performance Partially Connected Datastore Cluster Fully Connected Datastore Cluster  More datastores in cluster  better space and I/O balance  Larger datastore size  better space balance DRS Cluster DRS Cluster 32
  • 33. vSphere Storage Policy based Management Silver Disk Pool Gold Disk Pool Data store1 Data store2 Data store3 Data store4 Cluster-1 (Tier2 VMs) Cluster-2 (Tier1 VMs) Datastores with any storage profile Silver Disk Pool Gold Disk Pool Data store1 Data store2 Data store3 Data store4 Cluster-1 (Tier1 + Tier2 VMs) Datastores with identical storage profile 33
  • 34. Summary • Easier Storage Management • Effective storage capacity and performance management – Support for IO reservations – Integration with storage environments – Integration with vSphere solutions • Many exciting features available in vSphere 6.0 Beta! 34 https://communities.vmware.com/community/vmtn/vsphere-beta