QUESTION 231 An administrator is performing upgrades to ESXi 5.0 from ESX 3.x and ESX 4.x hosts.

What is true about this procedure? A. A direct, in-place upgrade can be performed only on ESX 4.1 and later hosts B. ESX 3.x and later hosts can be upgraded, but the upgrade cannot be performed in place C. A direct, in-place upgrade can be performed only on ESX 4.0 and later hosts D. A direct, in-place upgrade can be performed only on ESX 3.5 and later hosts

Answer: C
QUESTION 221 Which memory conservation technique allows the secure elimination of redundant copies of memory pages between virtual machines on the host? A. VMkernel Swap B. Memory Balloon Driver C. RAM Overcommit D. Transparent Page Sharing

Answer: D
QUESTION 211 An administrator would like to set up Host Profiles in a vSphere architecture. Using the vSphere Client, the administrator connects to vCenter Server and navigates to Management, then begins to build a new Host Profile but is unable to save the profile. Why is the administrator unable to save the completed profile? A. The administrator does not have the proper vSphere edition so this feature is not licensed B. The Host Profiles service is stopped and must be restarted to access this option C. No ESXi hosts have been added to vCenter Server, and this option will not be available until at least one host is managed D. No DRS/HA Cluster has been defined, and Host Profiles must be applied to a cluster.

Answer: A
QUESTION 201 Which of the following ports is not required by vCenter Server? A. 80 B. 902 C. 25 D. 443

Answer: C
QUESTION 191 An administrator is configuring iSCSI for use with a vSphere environment. Which networking feature should be configured on the ESXi host to improve iSCSI performance? A. Jumbo Frames B. NetQueue C. EtherChannel D. PortFast

Answer: A
QUESTION 181 An administrator decides to implement a Fixed multipath policy for an ESXi Host with 4 available paths and 4 configured vmfs datastores. How many paths will be used by host to send data to the 4 Datastores. (Choose One)

A. B. C. D.

4 8 16 12

Answer: A
QUESTION 171 A client wants to create a virtual machine with intense I/O traffic. Which type of disk should he choose? (choose two) A. Thick Provisioned B. Flat Disk C. Raw Mapped Device (RDM) D. Thin Provisioned

Answer: AC
QUESTION 161 When an existing role that has been assigned to users is removed from vCenter server, Which of the following can occur? (Choose Two) A. user or groups retain the removed role until they are manually assigned a new role B. users or groups can be reassigned to any available role C. users or groups that had privileges may no longer have any permissions in vCenter D. users or groups are automatically assigned to the next most restrictive role..

Answer: BC
QUESTION 151 An administrator attempts to increase the number of vCPUs in a virtual machine to 32. The configuration cannot be successfully completed. What are two likely causes of this issue? (Choose two) A. The vSphere 5 implementation has a license which does not include virtual SMP. B. The total number of cores in the host is 32 and a CPU must be reserved for ESXi. C. The virtual machine was built using version 7 virtual machine hardware. D. The vSphere 5 implementation is using an Enterprise license.

Answer: CD
QUESTION 141 What arc two functions of zoning in fibre Channel switches? (Choose two) A. Controls and isolates LUNs on a Fibre Channel storage array B. Restrict connections on a storage array to ESXi hosts that use the array C. Prevents non-ESXi hosts from possibly destroying VMFS data D. Reduces the number of targets and LUNs presented by a Fibre Channel storage array

Answer: CD
QUESTION 131 An administrator is working to implement Storage Profiles in their environment. Which two ways can storage capabilities be generated? (Choose two) A. Generated by a Storage Cluster while monitoring its LUNs B. Storage capabilities are pre-determined by the Storage Profile on creation C. Pulled from the array through the VMware APIs for Storage Awareness (VASA) D. Manually generated by the administrator

Answer: CD

QUESTION 121 The remote console performance for a virtual machine on an ESXi host appears to be degraded. What are two possible causes of the problem? (Choose two) A. The virtual NIC assigned to the virtual machine has a speed or duplex mismatch. B. The physical NIC assigned to the virtual machine port group containing the affected virtual machine has a speed or duplex mismatch. C. The physical NIC assigned to the Service Console port group has a speed or duplex mismatch. D. The physical NIC assigned to the VMkernel port group has a speed or duplex mismatch.

Answer: BD
QUESTION 111 An administrator is deploying several ESXi 5.x hosts. Centralized logging with accurate timestamp is a requirement for the deployment. Which two operations ensure these requirements are met? (Choose two.) A. Configure the host to use an NTP server B. Configure the hosts to direct logs to a syslog server C. Configure the host to record fame stamps m the local time zone D. Configure vilogger on the vSphere Management Appliance

Answer: AB
QUESTION 101 An administrator is creating a vNetwork Standard Switch with virtual machine and VMkernel networking. The administrator has two uplinks attached to the switch but wants to separate the virtual machine and VMKernel traffic to the two uplinks. Which load-balancing policy will guarantee this requirement is met, even if additional management or virtual machine traffic is added to the switch A. Route based on IP hash B. Router based on source MAC hash C. Route based on the originating port ID D. Use explicit failover

Answer: D
QUESTION 91 An administrator is enabling Enhanced vMotion Compatibility in a DRS cluster. The administrator wants the greatest degree of flexibility when adding new hosts in the cluster. Which setting satisfies this requirement? A. Any baseline compatible with the existing ESXi hosts in the cluster B. The highest baseline that is compatible with the ESXi hosts in the cluster C. The lowest baseline that is compatible with all ESXi hosts in the cluster D. The highest baseline that is compatible with vSphere 4.x and 5.x

Answer: C
QUESTION 81 Which two methods can be used to migrate a virtual machine from a vNetwork Standard to a vNetwork Distributed Switch? (Choose two) A. Edit the virtual adapter settings for the ESXi host and select a dvPort group from the list. B. Migrate the port group containing the virtual machine from a vNetwork Standard Switch using the Migrate Virtual machine networking option. C. Use the Migrate Virtual machine networking option to select the virtual machine from a list of

virtual machines. D. Edit the Network Adapter settings for the virtual machine and select a dvPort group from the list.

Answer: CD
QUESTION 71 A VMFS3 datastore has just been upgraded to VMFS5. In what two ways does this affect the partition format? (Choose two) A. The partition format is retained, which is the MBR (Master Boot Record) format. B. The partition format is retained, which is the MBR (Master Boot Record) format. To change the GPT (Guide Partition table) format, the datastore must be re-formatted. C. The partition format is retained, but will be upgraded to GPT (GUID Partition Table) format once the datastore exceeds 2TB. D. The partition format is upgraded to the GPT (GUID Partition table) format

Answer: AC
QUESTION 61 An administrator finds that vMotion and Storage VMotion operations do not succeed on a virtual machine. The virtual machine has been configured with N_Port ID Virtualization. The virtual machine has two data RDMs, one using a RAID 5 LUN on using a RAID0+1 LUN. The mapping file for RAID 5 LUN was created on the same datastore as the virtual machine, and the mapping file for the RAID0+1 LUN was placed in a datastore used for production data virtual disks. Which two statements are true about this configuration? (Choose two) A. vMotion cannot be used unless boot mapping files are placed on the same datastore. B. Storage vMotion cannot be used unless both mapping files are placed on the same datastore. C. Storage vMotion cannot be used with NPIV. D. vMotion cannot be used with NPIV

Answer: AC
QUESTION 51 What are two valid default upgrade baselines included with vSphere Update Manager? (Choose two) A. VMFS Upgrade to Match Host B. VMware Tools Upgrade to Match Host C. VA Upgrade to Latest D. Host Upgrade to Latest

Answer: BC
QUESTION 41 A company runs several instances of multi-threaded application that requires virtual machines to be configured with multiple vCPUs. Which metric is checked to determine if CPU contention exists on an ESXi 5.x host? A. %RUN B. %WAIT C. %USED D. %RDY

Answer: D
QUESTION 30 What is the traffic shaping adjustment? A. Minimum bandwidth adjusted in kbps

B. C. D.

Peak bandwidth adjusted in Mbps Burst time in seconds Burst size in KB

Answer: D
QUESTION 21 An administrator is attempting to deploy a virtual machine from an existing template, but deploy Virtual Machine from this template menu option is unavailable. What are two reasons this issue would occur? (Choose two) A. The template has not been stored in the proper format B. The vSphere Client is connected to an ESXi host C. The template is listed as Orphaned D. The administrator is using the vSphere Web Client

Answer: BC
QUESTION 11 A user attempts a remote SSH connection to a newly installed ESXi 5.x host to execute some commands. The SSH connection attempts fails, through the user is able to receive a ICMP ping back from the host. Which two connections might cause the issue? (Choose two) A. The shell timeout is set too low. B. The ParentRootLogin option is set to False. C. The SSH service is disabled on the host by default. D. The ESXi firewall blocks the SSH protocol by default.

Answer: CD
QUESTION 1 Which three modules are optional components that can be added from the vCenter Server installer? (Choose three.) A. VMware vCenter ESXi Dump Collector B. VMware vCenter Storage Monitoring C. VMware vSphere Web Client D. VMware vCenter Orchestrator E. VMware vSphere Update Manager

Answer: ACE

All the questions in the quiz along with their answers are shown below. Your answers are bolded. The correct answers have a green tick while the incorrect ones have a red cross. By default, where do hosts deployed with VMware Auto Deploy store logs?

In Memory First mounted LUN Local disk vMA

Which two users are assigned the Administrator role at the ESX Server level by default?

root, administrator root, vpxuser root, vpxa

root, hostd

Distributed Power Management (DPM) requires which technology to be available on the NIC?

Wake On LAN (WOL) DNS BMC NetBIOS

Your boss has asked you to setup 4 ESXi hosts to evaluate the new vSphere 5 release, which is the recommended install method to use in this situation?

Interactive Installation Scripted Installation vSphere Auto Deploy Installation Upgrade via VMware Update Manager

What is an HA Slot?

A slot is a logical representation of the memory and CPU resources that satisfy the requirements for any powered-on virtual machine in the cluster. A slot is a given to each Host in an HA Cluster, there is a maximum of 32 in an HA Cluster A slot is given to each Virtual Machine, this determines the restart priority. A slot has nothing to do with HA, it's how DRS calculates which Virtual Machine should be placed where.

What are the two iSCSI discovery methods that are supported by an ESXi host?

Static Targets, Dynamic Targets Static Discovery, SendTargets Static Discovery, FindTargets Dynamic Discovery, FindTargets

What protocol is used by an ESXi host to communicate with NAS devices?

NFS iSCSI CIFS SMB

What is an HA Slot?

A slot is a logical representation of the memory and CPU resources that satisfy the requirements for any powered-on virtual machine in the cluster. A slot is a given to each Host in an HA Cluster, there is a maximum of 32 in an HA Cluster

A slot is given to each Virtual Machine, this determines the restart priority. A slot has nothing to do with HA, it's how DRS calculates which Virtual Machine should be placed where.

Reference vSphere Availability Guide When is vNUMA enable by default on a virtual machines?

When a virtual machine has more than 8 vCPU's When the guestOS is configured using an SMP HAL All the time vNUMA is never enabled by default

Reference: Performance Best Practices for VMware vSphere 5.0 Which of the following actions would be LEAST likely to reduce resource contention?

Powering off a VM Enabling Swap-To-Cache Disable transparent memory page sharing Enabling vNUMA

Which of the following is a requirement for vCenter?

A Static IP address A DHCP IP address A valid (internal) domain name system (DNS) registration.

What the packaging format used by the VMware ESXi Image Builder?

.rar .zip .iso VIB

What are the three Host Isolation Response options available on an HA Cluster?

Shut down, Power off, Leave powered on Shut down, Power off, Leave powered off Shut down, Restart, Leave powered on Shut down, Restart, power on

Reference: vSphere Availability Guide To improve security which new feature has been added to ESXi?

Firewall Local Mode Anti-Virus vShield

The default security policy on a Port Group is:

Reject, Accept, Accept Reject, Reject, Reject Accept, Accept, Accept Reject, Reject, Accept

Which of the following formulas defines the amount of virtual machine memory that will always be composed of disk pages?

Memory allocated -(minus) memory limit Memory limit -(minus) memory reservation Memory limit -(minus) memory available Memory allocated -(minus) memory available

In an HA cluster after an initial election process, host are either:

Primary or Secondary Master or Slave King or Queen Live or Standby

Which feature of vSphere 5 can be used to reduce virtual machine memory overhead?

VMX Swap SplitRX Swap to Host Memory Reservations

Reference: Performance Best Practices for VMware vSphere 5.0 The organisation you work for is currently running vSphere 4.1 U1. Your team have just created a new Test/Dev environment based on vSphere 5. The VM management team have asked if you could combine the current 4.1 U1 environment and 5.0 vCenters with LinkedMode. What is the correct response to the VM management team?

vCenter 4.1 and vCenter cannot be joined with Linked-Mode Additional licensing is required to use Linked-Mode Only vCenter Administrators can use Linked-Mode No problem

Which new feature of vSphere 5 is used to collect host core dumps?

vMA 5.0 ESXi Dump Collector ESXi Syslog Collector VMware vDR 2.0

After an upgrade from ESX4.X to ESXi5.0 you notice that when you run the following command via the vMA: "esxcfg-vswif -l" you notice that their are no configure vswif interfaces listed. Why would this be?

The Service Console is no longer available with vSphere 5. All vswif interfaces are removed during the upgrade. During the upgrade process all vswif interfaces are migrated to vmk interfaces. During the upgrade process, vswif interfaces are disable by default, they'll need to be enable before they will be listed. The wrong command is being used. "vicfg-vswif -l" is the correct command to display all vswif interfaces.

Reference: VMware vSphere 5.0 Update Guide After an upgrade from ESX/ESXi 4.1 to ESXi 5.0, your monitoring department are reporting that they have stopped receiving Syslog data from all of the ESXi hosts. What could be the problem?

Custom ports that were opened by using the ESX/ESXi 4.1 esxcfg-firewall command do not remain open after the upgrade to ESXi 5.0. Custom ports that were opened by using the ESX/ESXi 4.1 esxcfg-firewall command do not do not get copied during the upgrade to ESXi 5.0. Tech Support Mode is disable by default after an upgrade to ESXi 5.0 stopping all traffic on all ports ESXi 5.0 cannot send logs to Syslog servers.

Reference: VMware vSphere 5.0 Update Guide After an upgrade from ESX4.X to ESXi5.0 you notice that your Port Group named "Service Console" is missing. Where has it gone?

Because ESXi 5.0 has no Service Console, upgrading from ESX 4.x to ESXi 5.0 removes the Service Console port group. During the upgrade all Port Groups are removed. The Service Console Port Group will need to be recreated. During the upgrade all Port Groups are moved to Distributed Switches. During the upgrade all Port Groups are moved to Standard Switches.

Reference: VMware vSphere 5.0 Update Guide When deploying hosts with VMware Auto Deploy, which is the recommended method to configure ESXi once it has been installed?

Host Profiles PowerCLI Manually via Direct Console

You boss has asked you deploy 40 ESXi hosts as quickly as possible, which is the recommended install method to use in this situation?

Interactive Installation Scripted Installation VMware Auto Deploy Installation Upgrade via VMware Update Manager

After an upgrade from ESXi4.0 to ESXi5.0 you are unable to putty to your hosts. What could have caused this?

SSH configuration is not migrated for ESX 4.x hosts or ESXi 4.0 hosts. For these hosts, SSH access is disabled during the upgrade or migration process SSH is not available with ESXi 5.0, it is only available with ESX 5.0. SSH is always disabled after an upgrade or clean installation. The SSH port in ESXi 5.0 has changed to 443 for added security.

Reference: VMware vSphere 5.0 Update Guide You have been asked to provide high availability for your vCenter server. Which product would you use to help achieve this?

Fault-Tolerance vCenter Linked-Mode vCenter Heartbeat Microsoft Clustering Services

Which of the following is no longer available with vSphere 5?

ESX vCenter vSphere Client PowerCLI

Which if the following is not a supported location for a host diagnostic partition?

Shared local storage Private local storage Private SAN storage Shared SAN storage

VMware vSphere can be evaluated prior to purchase. What is the maximum number of days the evaluation can be used before a license must be purchased?

30 Days 60 Days 90 Days 120 Days

ESXi 5.0 supports only LAHF and SAHF CPU instructions?

True False

Which of the following installation methods installs the ESXi image directly into the Host memory?

Interactive ESXi Installation Scripted ESXi Installation vSphere Auto Deploy ESXi Installation Option Upgrade via VMware Update Manager

What are the 4 VM Restart Priority options available on an HA Cluster?

Disabled, Low, Medium, High Enabled, Small, Medium, Large 1st, 2nd, 3rd, 4th Priority, 2nd Tier, 3rd Tier, Best Effort

Reference: vSphere Availability Guide Which feature of vSphere 5 can be used to increase network throughput to virtual machines?

SplitRX NPIV VMDirectpath I/O RDM

Reference: Performance Best Practices for VMware vSphere 5.0 Which new version of VMFS is introduced by vSphere 5?

VMFS-2 VMFS-3 VMFS-5 VMFS-4

The upgrade procedure from VMFS-3 to VMFS-5 will require downtime of that datastore?

False True

Which of the following is not a benefit of VMware Auto Deploy?

Decouples the VMware ESXi host from the physical server and eliminates the boot disk Eliminates configuration drift Simplifies patching and updating Accelerates deployment of virtual machines

Your colleague has accidentally allocated more vRAM than your company are licensed for. What will happen to your virtual machines?

All VM's will be Powered Off

New VM's can not be Powered On VMware will be notified Nothing will happen

What are the Automation Levels on a DRS Cluster?

Manual, Partially Automatic, Fully Automatic Manual, Partially Automated, Fully Automated Manual, Semi Automated, Fully Automatic Manual, Semi Automatic, Automatic

Reference: VMware DRS Which of the following dvPort binding types have been removed in vSphere 5?

Dynamic Binding Ephemeral Binding Static Binding

Reference: Choosing a dvPort binding type What are the three default roles provided on an ESXi host?

No Access, Read Only and Administrator Read only, Operator Access and Administrator Virtual Machine User, Virtual Machine Power User and Administrator Network Consumer, Datastore Consumer and Resource Pool Administrator

Which of the following actions is not available via the Direct Console?

Shutdown host Enter host into Maintenance Mode View host logs Configure host DNS

It is possible to Storage vMotion virtual machines that have snapshots?

True False

A memory reservation defines:

the amount of virtual machine memory that can be paged to the VMkernel swapfile the amount of physical memory that is guaranteed to the VM the maximum amount of physical memory that will be used by a VM the amount of host memory reserved for the VMkernel

Which of the following is not a new feature made available with vSphere 5?

sDRS

vDR VSA vSphere Web Client

How has vMotion been improved in vSphere 5?

Multiple vMotion vmknics, allowing for more and faster vMotion operations Long distance vMotion, allowing for vMotions over a large geographical area Multi-platform vMotion, allowing for vMotions between vSphere and Hyper-V platforms Storage vMotion, allowing for virtual machines storage to be moved between datastores

VMFS-5 upgraded from VMFS-3 continues to use the previous file block size which may be larger than the unified 1MB file block size.

True False

Which new feature of vSphere 5 is used to create ESXi installation images with a customized set of updates, patches, and drivers?

Image Builder Host Profiles Auto Deploy esxcli

What is the name of the globally unique identifier assigned to each Fibre Channel Port?

IP Address MAC Address (Mac) World Wide Name (WWN) Port_ID

To get the maximum performance benefits of vNUMA it is recommended to:

Make sure your clusters are composed entirely of hosts with matching NUMA architecture. Only use single vCPU virtual machines Enable EVC on your clusters Disable Hyper-Threading

Which vSphere feature provides dynamic allocation of storage capacity?

vStorage APIs / VCB Thinapp sDRS vStorage Thin Provisioning

All the questions in the quiz along with their answers are shown below. Your answers are bolded. The correct answers have a green tick while the incorrect ones have a red cross. What is the Maximum Virtual NICs per virtual machine?

2 4 8 10

What is the Maximum Virtual machine swap file size?

255GB 256GB 2TB 1TB

What is the Maximum VMDirectPath PCI/PCIe devices per virtual machine?

2 3 8 6, if 2 of them are Teradici devices

What is the Maximum USB controllers per virtual machine?

2 3 4 1

What is the Maximum RAM per virtual machine?

255GB 256GB 2TB 1TB

What is the Maximum Floppy controllers per virtual machine?

2 4 3 1

What is the Maximum Concurrent remote console connections to a virtual machine?

Unlimited 10 8

40

What is the Maximum Virtual SCSI targets per virtual SCSI adapter?

4 60 14 15

What is the Maximum Virtual Disks per virtual machine (PVSCSI)?

10 4 1 60

What is the Maximum USB 3.0 devices per virtual machine?

3 20 10 1

What is the Maximum IDE devices per virtual machine?

1 128 16 4

What is the Maximum Video memory per virtual machine?

265MB 64MB 1GB 128MB

What is the Maximum IDE controllers per virtual machine?

2 3 4 1

What is the Maximum Virtual CPUs per virtual machine (Virtual SMP) ?

8 4 16 32

What is the Maximum Virtual SCSI targets per virtual machine?

15 4 65 60

What is the Maximum Virtual SCSI adapters per virtual machine?

1 2 3 4

What is the Maximum Serial ports per virtual machine?

1 10 3 4

What is the Maximum Parallel ports per virtual machine?

1 2 6 3

What is the Maximum xHCI USB controllers per virtual machine?

1 3 10 20

What is the Maximum USB devices connected to a virtual machine?

1 6 10 20

What is the Maximum Floppy devices per virtual machine?

1 3 4 2

What is the Maximum virtual machine Disk Size?

256GB 1TB 2TB 2TB minus 512B

Choosing a dvPort binding type

Details When choosing a dvPort binding type, consider how you want to connect your virtual machines and virtual network adapters to a vDS and how you intend to use your virtual machines. Port binding type, along with all other vDS and dvPort group configuration, can be set only through vCenter Server. Solution

Types of port binding

These three different types of port binding determine when dvPorts in a dvPort group are assigned to virtual machines:

Static Binding Dynamic Binding Ephemeral Binding

Static binding When you connect a virtual machine to a dvPort group configured with static binding, a dvPort is immediately assigned and reserved for it, guaranteeing connectivity at all times. The dvPort is disconnected only when the virtual machine is removed from dvPort group. You can connect a virtual machine to a static binding dvPort group only through vCenter Server. Starting vSphere 5.0 'Auto_Expand' property is turned on by default for Static Port Binding. This property allows a dvPortgroup to expand automatically by a small predefined margin every time dvPortgroup is about to run out of dvPorts. Note: Static binding is the default setting, recommended for general use. Dynamic binding In a dvPort group configured with dynamic binding, a dvPort is assigned to a virtual machine only when the virtual machine is powered on and its NIC is in a connected state. The dvPort is freed when the virtual machine is powered off or the virtual machine's NIC is disconnected. Virtual machines connected to a dvPort group configured with dynamic binding must be powered on and off through vCenter. Dynamic binding can be used in environments where you have more virtual machines than available ports, but do not plan to have a greater number of virtual machines active than you have available ports. For example, if you have 300 virtual machines and 100 ports, but never

have more than 90 virtual machines active at one time, then dynamic binding would be appropriate for your dvPort group. Note: Dynamic binding is deprecated in ESXi 5.0. Ephemeral binding In an dvPort group configured with ephemeral binding, a dvPort is created and assigned to a virtual machine when the virtual machine is powered on and its NIC is in a connected state. The dvPort is deleted when the virtual machine is powered off or the virtual machine's NIC is disconnected. Ephemeral dvPort assignments can be made through ESX/ESXi as well as vCenter, giving you the flexibility to manage virtual machine connections through the host when vCenter is down. Although only ephemeral binding allows you to modify virtual machine network connections when vCenter is down, network traffic is unaffected by vCenter failure regardless of port binding type. Note: Ephemeral dvPortgroups should be used only for recovery purposes when you want to provision dvPorts directly on host bypassing vCenter Server, not for any other case. This is true for several reasons:

Scalability An ESX/ESXi 4.x host can support up to 1016 ephemeral dvPortgroups and an ESXi 5.x host can support up to 256 ephemeral dvPortgroups . Since ephemeral dvPortgroups are always pushed to hosts, this effectively is also the vCenter Server limit. For more information, see Configuration Maximums for VMware vSphere 5.0 and Configuration Maximums for VMware vSphere 4.1.

Performance Every operation, including add-host and virtual machine power operation, is slower comparatively because ports are created/destroyed in the operation code path. Virtual machine operations are far more frequent than host-add or switch-operations, so ephemeral dvPorts are more demanding in general.

Non-persistent (that is, "ephemeral") ports Port-level permissions and controls are lost across power cycles, so no historical context is saved.

ESXTOP Intro Thresholds Howto Run Howto Capture Howto Analyze Howto Limit esxtop to specific VMs

References Changelog This page is solely dedicated to one of the best tools in the world for ESX; esxtop.

Intro
I am a huge fan of esxtop! I read a couple of pages of the esxtop bible every day before I go to bed. Something I however am always struggling with is the thresholds of specific metrics. I fully understand that it is not black/white, performance is the perception of a user in the end. There must be a certain threshold however. For instance it must be safe to say that when %RDY constantly exceeds the value of 20 it is very likely that the VM responds sluggish. I want to use this article to define these thresholds, but I need your help. There are many people reading these articles, together we must know at least a dozen metrics lets collect and document them with possible causes if known. Please keep in mind that these should only be used as a guideline when doing performance troubleshooting! Also be aware that some metrics are not part of the default view. You can add fields to an esxtop view by clicking f on followed by the corresponding character. I used VMworld presentations, VMware whitepapers, VMware documentation, VMTN Topics and of course my own experience as a source and these are the metrics and thresholds I came up with so far. Please comment and help build the main source for esxtop thresholds.

Metrics and Thresholds

Display CPU Metric %RDY Threshold Explanation Overprovisioning of vCPUs, excessive usage of vSMP 10 or a limit(check %MLMTD) has been set. See Jasons explanation for vSMP VMs Excessive usage of vSMP. Decrease amount of vCPUs 3 for this particular VM. This should lead to increased scheduling opportunities. The percentage of time spent by system services on behalf of the world. Most likely caused by high IO 20 VM. Check other metrics and VM for possible root cause The percentage of time the vCPU was ready to run but deliberately wasnt scheduled because that would 0 violate the CPU limit settings. If larger than 0 the world is being throttled due to the limit on CPU. VM waiting on swapped pages to be read from disk. 5 Possible cause: Memory overcommitment. If larger than 0 host is forcing VMs to inflate balloon 1 driver to reclaim memory as host is overcommited. 1 If larger than 0 host has swapped memory pages in the

CPU

%CSTP

CPU

%SYS

CPU

%MLMTD

CPU MEM MEM

%SWPWT MCTLSZ SWCUR

MEM MEM MEM MEM MEM

SWR/s SWW/s

1 1

CACHEUSD 0 ZIP/s UNZIP/s 0 0

MEM

N%L

80

NETWORK %DRPTX NETWORK %DRPRX DISK DISK DISK DISK GAVG DAVG KAVG QUED

1 1 25 25 2 1

DISK

ABRTS/s

DISK DISK

RESETS/s CONS/s

1 20

past. Possible cause: Overcommitment. If larger than 0 host is actively reading from swap(vswp). Possible cause: Excessive memory overcommitment. If larger than 0 host is actively writing to swap(vswp). Possible cause: Excessive memory overcommitment. If larger than 0 host has compressed memory. Possible cause: Memory overcommitment. If larger than 0 host is actively compressing memory. Possible cause: Memory overcommitment. If larger than 0 host has accessing compressed memory. Possible cause: Previously host was overcommited on memory. If less than 80 VM experiences poor NUMA locality. If a VM has a memory size greater than the amount of memory local to each processor, the ESX scheduler does not attempt to use NUMA optimizations for that VM and remotely uses memory via interconnect. Dropped packets transmitted, hardware overworked. Possible cause: very high network utilization Dropped packets received, hardware overworked. Possible cause: very high network utilization Look at DAVG and KAVG as the sum of both is GAVG. Disk latency most likely to be caused by array. Disk latency caused by the VMkernel, high KAVG usually means queuing. Check QUED. Queue maxed out. Possibly queue depth set to low. Check with array vendor for optimal queue depth value. Aborts issued by guest(VM) because storage is not responding. For Windows VMs this happens after 60 seconds by default. Can be caused for instance when paths failed or array is not accepting any IO for whatever reason. The number of commands reset per second. SCSI Reservation Conflicts per second. If many SCSI Reservation Conflicts occur performance could be degraded due to the lock on the VMFS.

Running esxtop
Although understanding all the metrics esxtop provides seem to be impossible using esxtop is fairly simple. When you get the hang of it you will notice yourself staring at the metrics/thresholds more often than ever. The following keys are the ones I use the most.

Open console session or ssh to ESX(i) and type:

esxtop

By default the screen will be refreshed every 5 seconds, change this by typing:
s 2

Changing views is easy type the following keys for the associated views:
c m n i d u v p = = = = = = = = cpu memory network interrupts disk adapter disk device (includes NFS as of 4.0 Update 2) disk VM power states

V = only show virtual machine worlds e = Expand/Rollup CPU statistics, show details of all worlds associated with group (GID) k = kill world, for tech support purposes only! l = limit display to a single group (GID), enables you to focus on one VM # = limiting the number of entitites, for instance the top 5 2 8 4 e 6 = = = = = highlight a row, moving down highlight a row, moving up remove selected row from view statistics broken down per world statistics broken down per world

Add/Remove fields:
f <type appropriate character>

Changing the order:

o <move field by typing appropriate character uppercase = left, lowercase = right>

Saving all the settings youve changed:

W

Keep in mind that when you dont change the file-name it will be saved and used as default settings. Help:
?

In very large environments esxtop can high CPU utilization due to the amount of data that will need to be gathered and calculations that will need to be done. If CPU appears to highly

utilized due to the amount of entities (VMs / LUNs etc) a command line option can be used which locks specific entities and keeps esxtop from gathering specific info to limit the amount of CPU power needed:
esxtop -l

More info about this command line option can be found here.

Capturing esxtop results

First things first. Make sure you only capture relevant info. Ditch the metrics you dont need. In other words run esxtop and remove/add(f) the fields you dont actually need or do need! When you are finished make sure to write(W) the configuration to disk. You can either write it to the default config file(esxtop4rc) or write the configuration to a new file. Now that you have configured esxtop as needed run it in batch mode and save the results to a .csv file:
esxtop -b -d 2 -n 100 > esxtopcapture.csv

Where -b stands for batch mode, -d 2 is a delay of 2 seconds and -n 100 are 100 iterations. In this specific case esxtop will log all metrics for 200 seconds. If you want to record all metrics make sure to add -a to your string. Or what about directly zipping the output as well? These .csv can grow fast and by zipping it a lot of precious diskspace can be saved!
esxtop -b -a -d 2 -n 100 | gzip -9c > esxtopoutput.csv.gz

Analyzing results
You can use multiple tools to analyze the captured data. 1. perfmon 2. excel 3. esxplot Lets start with perfmon as Ive used perfmon(part of Windows also know as Performance Monitor) multiple times and its probably the easiest as many people are already familiar with it. You can import a CSV as follows: 1. 2. 3. 4. 5. 6. 7. 8. Run: perfmon Right click on the graph and select Properties. Select the Source tab. Select the Log files: radio button from the Data source section. Click the Add button. Select the CSV file created by esxtop and click OK. Click the Apply button. Optionally: reduce the range of time over which the data will be displayed by using the sliders under the Time Range button.

9. Select the Data tab. 10. Remove all Counters. 11. Click Add and select appropriate counters. 12. Click OK. 13. Click OK. The result of the above would be:

With MS Excel it is also possible to import the data as a CSV. Keep in mind though that the amount of captured data is insane so you might want to limit it by first importing it into perfmon and then select the correct timeframe and counters and export this to a CSV. When you have done so you can import the CSV as follows: 1. 2. 3. 4. 5. 6. 7. 8. Run: excel Click on Data Click Import External Data and click Import Data Select Text files as Files of Type Select file and click Open Make sure Delimited is selected and click Next Deselect Tab and select Comma Click Next and Finish

All data should be imported and can be shaped / modelled / diagrammed as needed. Another option is to use a tool called esxplot. You can download the latest version here.

1. 2. 3. 4.

Run: esxplot Click File -> Import -> Dataset Select file and click Open Double click host name and click on metric

As you can clearly see in the screenshot above the legend(right of the graph) is too long. You can modify that as follows: 1. Click on File -> preferences 2. Select Abbreviated legends 3. Enter appropriate value For those using a Mac, esxplot uses specific libraries which are only available on the 32Bit version of Python. In order for esxplot to function correctly set the following environment variable:
export VERSIONER_PYTHON_PREFER_32_BIT=yes

Limiting your view

In environments with a very high consolidation ratio (high number of VMs per host) it could occur that the VM you need to have performance counters for isnt shown on your screen. This happens purely due to the fact that height of the screen is limited in what it can display. Unfortunately there is currently no command line option for esxtop to specify specific VMs that need to be displayed. However you can export the current list of worlds and import it again to limit the amount of VMs shown.
esxtop -export-entity filename

Now you should be able to edit your file and comment out specific worlds that are not needed to be displayed.
esxtop -import-entity filename

I figured that there should be a way to get the info through the command line as and this is what I came up with. Please note that <virtualmachinename> needs to be replaced with the name of the virtual machine that you need the GID for.

VMWID=`vm-support -x | grep <virtualmachinename> |awk '{gsub("wid=", "");print $1}'` VMXCARTEL=`vsish -e cat /vm/$VMWID/vmxCartelID` vsish -e cat /sched/memClients/$VMXCARTEL/SchedGroupID

Now you can use the outcome within esxtop to limit(l) your view to that single GID. William Lam has written an article a couple of days after I added the GID section. The following is a lot simpler than what I came up with, thanks William!
VM_NAME=STA202G ;grep "${VM_NAME}" /proc/vmware/sched/drm-stats '{print $1}' | awk

References
The following documents / articles have been used as a reference:

Interpreting esxtop Statistics Performance Troubleshooting for VMware vSphere 4 and ESX 4.0 Hypervisor.fr Easter Eggs esxtop Performance Training Using Perfmon for esxtop esxplot download page esxplot remove vertical lines from perfmon ESXtop for Advanced Users (2008) ESXtop for Advanced Users (2009) ESXtop for Advanced Users (2010) limiting esxtop to specific VMs

Changelog
07-01-2010 | decreased %RDY from 20 to a value of 10 22-01-2010 | added CPU > TIMER/S 22-01-2010 | added MEM > N%L 24-01-2010 | added sections (howto) 02-02-2010 | expanded analyze section and included screenshots 10-02-2010 | decreased %CSTP from 100 to 5 10-02-2010 | decreased KAVG from 5 to 2 23-03-2010 | increase %SWPWT from 1 to 5 23-03-2010 | added e, V, i, 2, 4, 6, 8 in the views section 16-06-2010 | added -l functionality and stressed NFS added option 08-11-2010 | added l, e, l, #, %SYS, ZIP/s, UNZIP/s, CACHEUSD 11-11-2010 | added threshold for CONS/s 11-11-2010 | Redid some of the formatting 11-11-2010 | added limiting the view section

Using esxtop to identify storage performance issues

Details
This article provides information about esxtop and latency statistics that can be used when troubleshooting performance issues with SAN-connected storage (Fibre Channel or iSCSI).

Solution
The interactive esxtop utility can be used to provide I/O metrics over various devices attached to a VMware ESX host.

Configuring monitoring using esxtop

To monitor storage performance per HBA:

1. 2. 3. 4. 5. 6. 7.

Start esxtop by typing esxtop at the command line. Press d to switch to disk view (HBA mode). Press f to modify the fields that are displayed. To view the entire Device name, press SHIFT + L and enter 38 in Change the name field size. Press b, c, d, e, h, and j to toggle the fields and press Enter. Press s, then 2 to alter the update time to every 2 seconds and press Enter. See Analyzing esxtop columns for a description of relevant columns.

Note: The following options are only available in VMware ESX 3.5 and later. To monitor storage performance on a per-LUN basis:

1. 2. 3. 4. 5. 6.

Start esxtop by typing esxtop from the command line. Press u to switch to disk view (LUN mode). Press f to modify the fields that are displayed. Press b, c, f, and h to toggle the fields and press Enter. Press s, then 2 to alter the update time to every 2 seconds and press Enter. See Analyzing esxtop columns for a description of relevant columns.

For more information, see How to increase the width of the device field in esxtop to show the complete naa id (1035989). To monitor storage performance on a per-virtual machine basis:

1. 2. 3. 4. 5. 6.

Start esxtop by typing esxtop at the command line. Type v to switch to disk view (virtual machine mode). Press f to modify the fields that are displayed. Press b, d, e, h, and j to toggle the fields and press Enter. Press s, then 2 to alter the update time to every 2 seconds and press Enter. See Analyzing esxtop columns for a description of relevant columns.

Analyzing esxtop columns

This table lists the relevant columns and a brief description of these values: Column CMDS/s Description This is the number of IOPS (Input/Output Operations Per Second) being sent to or coming from the device or virtual machine being monitored

DAVG/cmd This is the average response time in milliseconds per command being sent to the device KAVG/cmd This is the amount of time the command spends in the VMkernel GAVG/cmd This is the response time as it is perceived by the guest operating system. This number is calculated with the formula: DAVG + KAVG = GAVG

These columns are for both reads and writes, whereas xAVG/rd is for reads and xAVG/wr is for writes. The combined value of these columns is the best way to monitor performance, but high read or write response time it may indicate that the read or write cache is disabled on the array. All arrays perform differently, howeverDAVG/cmd, KAVG/cmd, and GAVG/cmd should not exceed more than 10 milliseconds (ms) for sustained periods of time. Note: VMware ESX 3.0.x does not include direct functionality to monitor individual LUNs or virtual machines using esxtop. Inactive LUNs lower the average for DAVG/cmd, KAVG/cmd, and GAVG/cmd. These values are also visible from the vCenter Server performance charts. For more information, see Performance Charts in the Basic System Administration Guide. If you experience high latency times, investigate current performance metrics and running configuration for the switches and the SAN targets. Check for errors or logging that may suggest a delay in operations being sent to, received, and acknowledged. This includes the array's ability to process I/O from a spindle count aspect, or the array's ability to handle the load presented to it. If the response time increases to over 5000 ms (or 5 seconds), VMware ESX will time out the command and abort the operation. These events are logged; abort messages and other SCSI errors can be reviewed inthe following logs: ESX /var/log/vmkernel ESXi /var/log/messages The type of storage logging you may see in these files depends on the configuration of the server. You can find the value of these options by navigating to Host > Configuration > Advanced Settings > SCSI > SCSI.Log* or SCSI.Print*.

VMware ESXi and ESX Hypervisor FAQs

What is the difference between VMware ESX and VMware ESXi? VMware ESX and VMware ESXi are both bare-metal hypervisor architectures that install directly on the server hardware. Both provide industry-leading performance and scalability; the difference resides in the architecture components and the operational management of VMware ESXi. Although neither hypervisor architectures relies on an OS for resource management, VMware ESX relies on a Linux operating system, called the service console, to perform two management functions: executing scripts and installing third party agents for hardware monitoring, backup or systems management. The service console has been removed from ESXi, drastically reducing the hypervisor footprint and completing the ongoing trend of migrating management functionality from the local command line interface to remote management tools. The smaller code base of ESXi represents a smaller attack surface and less code to patch, improving reliability and security. The functionally of the service console is replaced by remote command line interfaces and adherence to system management standards. View a detailed comparison of VMware ESX and VMware ESXi management. Can I deploy VMware vSphere 5 using ESX? Starting with VMware vSphere 5, ESXi is only hypervisor option available to deploy vSphere. VMware ESXis smaller code base and lack of reliance on a general purpose operating system make it the more reliable and secure choice. Its compact design allows it to be embedded in mainstream physical servers for simpler and faster deployments. VMware ESXi was also designed with simplicity in mind. Its menu-driven startup and automatic configurations make it the easiest way to get started with VMware virtualization. VMware recommends that customers migrate existing ESX deployment to ESXi when upgrading to vSphere 5 or vSphere 4.1.Before migrating you should consider the following points before deploying ESXi or switching from ESX to ESXi:

Make sure your hardware is supported on ESXi as the hardware compatibility list (HCL) is not identical to ESX. In general, VMware ESX has a larger HCL due to its support for legacy servers but expect support parity for ESXi and ESX on all new servers. Make sure that your third party management and back up tools are compatible with ESXi. Some of these tools have a dependency on the service console (COS) of ESX, which has been removed from the architecture of ESXi. VMware has developed alternative integration points for ESXi and is actively engaged with backup and management partners to help them make the transition from ESX to ESXi. Most major management or backup vendors such as IBM, BMC, CA, HP, EMC, Symantec and Veritas offer ESXi-compatible products but you should check with your third party vendor for the latest information on supported products and version numbers. Read this KB article for a detailed comparison of VMware ESXi and VMware ESX management.

Does VMware still support ESX 4.x or older? VMware will continue to provide technical support for VMware ESX according to the VMware vSphere support policy. What type of hardware is required to run vSphere with the ESXi architecture? The VMware ESXi hypervisor architecture is certified on over 1000 server systems and can run off certified USB keys on certain models. In general, new hardware systems supported for ESX will also be supported for ESXi, but please search the hardware compatibility lists (HCLs) for information about a specific hardware system. What Operating Systems can I run on vSphere with the ESXi architecture? VMware offers the broadest operating system support as compared to any other virtualization vendor. VMware ESXi and ESX have virtually the same guest operating system support. Please search the guest operating system compatibility list for a full list of supported guest operating systems. Can I run virtual machines created by Microsoft Virtual Server, Microsoft Virtual PC, or VMware Server on VMware ESX or ESXi? Yes. You can use the free VMware vCenter Converter to import virtual machines that were created using VMware Server, Microsoft Virtual Server, or Microsoft Virtual PC version 7 or higher. VMware vCenter Converter also supports conversions from sources such as physical machines and certain third party disk image formats. View the VMware vCenter Converter FAQs for further details. How is VMware ESXi different than VMware vSphere? ESXi is VMwares most advanced bare metal hypervisor architecture. It is the only hypervisor architecture available with vSphere 5 or later. Customers install the ESXi software on a physical sever to virtualize its resources (CPU, memory, etc.) and share them among virtual machines. Applying vSphere licenses (Essentials, Essentials Plus, Standard, Enterprise, Enterprise Plus) transforms a collection of individual logically separate ESXi hosts into a pool of virtualized resources that can be dynamically allocated to virtual machines. vSphere licenses also allow customers to take advantage of features such as vMotion, DRS, HA, etc. What is the difference between VMware vSphere Hypervisor and VMware vSphere? VMware vSphere Hypervisor is the entry level edition of VMware vSphere. It is available at no cost in order to help companies of all sizes experience the fundamental benefits of virtualization. VMware vSphere Hypervisor enables customers to consolidate physical servers running multiple virtual machines to a single vSphere host. VMware vSphere Hypervisor is ideal for IT professionals to become familiar with virtualization technology and prove its value in their own organizations. In addition to server consolidation capabilities, the VMware vSphere editions available for purchase

(Essentials, Essentials Plus, Standard, Enterprise, Enterprise Plus) offer more advanced capabilities, such as high availability, live migration, power management, automatic load balancing and much more. To unlock such capabilities customers need to purchase the corresponding VMware vSphere license and deploy VMware vCenter Server management software which enables centralized management of multiple vSphere hosts. How can I evaluate the VMware ESXi hypervisor architecture? The simplest way to evaluate the VMware ESXi architecture is to download and install vSphere Hypervisor. Alternatively you can leverage the vSphere evaluations Will VMware Update Manager Patch my vSphere host when using the ESXi architecture? VMware vCenter Update Manager will update ESXi An update utility is also provided for environments where VMware Update Manager is not present. Note:VMware vCenter Update Manager is not available for the free VMware vSphere Hypervisor. Does vSphere maintain log files or integrate with syslog when using the ESXi architecture? Yes, using the ESXi architecture you can generate log files Since ESXi 4.1, persistent logging is the default configuration. You can configure ESXi to send logs to a file on a datastore as well as to a syslog server. For more information see the ESXi Management page. Do the vCLI and PowerCLI work the same with both ESX and ESXi? Most of the vCLI and PowerCLI commands work the same on both platforms. There are a few vCLI commands that are not applicable to ESX; these are outlined in Table 1-5 of the vCLI Reference Guide. There are also some specific differences in behavior between some vCLI commands and the equivalent console OS commands; these are outlined in KB1008194. What is Tech Support Mode? Tech Support Mode is a command-line interface for diagnosis and repair of an ESXi system. It is enabled but inactive for default installations. Starting with ESX 4.1, Tech Support Mode can be enabled and disable through the DCUI or through vCenter Server, and can also be accessed remotely via SSH. For more information, see the ESXi Management page. To learn how Tech Support Mode works in ESXi 5.0, please refer to KB1003677. Is SSH supported on ESXi? SSH to Tech Support Mode on an ESXi system is supported since version 4.1. How do I run esxtop on ESXi? Esxtop functionality is provided by the vCLI command resxtop, which has the same features as esxtop in the console OS. How do I kill a stuck VM when using the ESXi architecture? Since vSphere 4.1, the vCLI has been enhanced to include a command for killing stuck VMs. Instructions for how to kill a stuck VM in ESXi 4.0 can be found in KB1014165. How do I monitor the hardware on vSphere host when using ESXi hypervisor architecture? ESXi supports both has CIM APIs and SNMP v2 for enabling hardware monitoring without installing agents. Hardware can be monitored via the vCenter console and via the major hardware monitoring tools from OEM vendors. See the ESXi Management page for more details.

ESXTOP to the rescue - VM latency

Earlier on I have mostly used ESXTOP for basic troubleshooting reasons such as CPU ready and the like. Last weekend we had a major incident which was caused by a power outage which affected a whole server room. After the power was back on we had a number VMs that was showing very poor performance - as in it took about one hour to log in to Windows. It was quite random which VMs it was. The ESX hosts looked fine. After a bit of troubleshooting the only common denominator was that the slow VMs all resided on the same LUN. When I contacted the storage night duty the response was that there was no issue on the storage system. I was quite sure that the issue was storage related but I needed some more data. The hosts were running v3.5 so troubleshooting towards storage is not easy. I started ESXTOP to see if I could find some latency numbers. I found this excellent VMware KB article which pointed me in the right direction. For VM latency, start ESXTOP and press 'v' for VM storage related performance counters. The press 'f' to modify counters shown, then press 'h', 'i', and 'j' to toggle relevant counters (see screendump 2) - which in this case is latency stats (remember to stretch the window to see all counters) What I found was that all affected VMs had massive latency towards the storage system for DAVG/cmd (see screendump 1) of about 700 ms (rule of thumb is that max latency should be about 20 ms). Another important counter is KAVG/cmd which is time commands spend in the VMkernel, the ESX host, (see screendump 3). So there was no latency in the ESX host and long latency towards the storage system. After pressing the storage guys for a while, they had HP come take a look at it, and it turned out that there was a defect fiber port in the storage system. After this was replaced everything worked fine and latency went back to nearly zero. In this case, it was only because I had proper latency data from ESXTOP that I could be almost certain that the issue was storage related.

Screendump 1

Screendump 2

Screendump 3