CPU

Scheduling virtual CPUs

Recently someone asked me what VMware scheduling is, so I thought I would cover that in this blog post. Scheduling, or virtual CPU scheduling, happens behind the scenes and is not a very visible component of virtualization, but is absolutely critical for virtualization to work properly. You should have at least a basic understanding of how it works so you understand how it impacts virtual machine (VM) performance and what to look for when troubleshooting malfunctions.

Scheduling 101

The scheduler is a component of the VMkernel that schedules requests for the virtual CPUs assigned to virtual machines to the physical CPUs of the host server. Whenever a virtual machine (VM) uses its virtual CPU, the VMkernel has to find a free physical CPU (or core) for the VM to use. On a typical host server, the number of virtual CPUs usually outnumbers the number of physical CPUs, so the VMs are all competing to use the limited number of physical CPUs that the host has. The scheduler’s job is to find CPU time for all the VMs that are requesting it and to do it in a balanced way, so performance for any one VM does not suffer. This is not always an easy task, especially when VMs are assigned multiple virtual CPUs (virtual symmetric multiprocessing, or vSMP) as this further complicates the scheduling.

Continued »

Lord of the virtual rings: Why hardware matters to your VMs

x86 computer operating systems utilize protection rings that provide a range of protection levels called rings in which code can execute. These rings are arranged in a hierarchical manner from the most privileged (most trusted, Ring 0) to least privileged (least trusted, Ring 3) as shown below.

The enforcement of these rings is done by the processor (CPU) which uses different operating modes that place restrictions on the operations that can be performed by the process currently running in the CPU. Ring 0 has the highest level privilege and is where the operating system kernel normally runs. Code executing in Ring 0 is referred to as running in kernel mode, which is also known as privileged or supervisor mode. All other code such as applications running on the operating system operate in less privileged rings, typically Ring 3. With non-virtualized systems, the operating system runs in privileged mode in Ring 0 and owns the server hardware, applications run in Ring 3 with less privileges as depicted below.

On virtualized systems the hypervisor or Virtual Machine Monitor (VMM) runs in privileged mode in Ring 0 and the VM’s guest operating system must instead operate in Ring 1 as depicted below.

This can cause problems however because most VM guest operating systems are designed to run in Ring 0. To overcome this, the VMM fools the VM’s guest operating systems into thinking they are running in Ring 0 by trapping privileged instructions and emulating them by the VMM. This emulation causes a slight bit of overhead and is the reason that VM performance can typically only achieve up to 98% of native performance compared to physical servers. To overcome this, newer CPUs like the AMD-V and Intel-VT have features that were specifically designed for virtualization and use a new privilege level called Ring -1 (minus one) for the VMM to reside in as depicted below.

This allows for better performance as the VMM no longer needs to fool the VM guest operating system into thinking that it is running in Ring 0 as it can run in there without conflicting with the VMM which has moved to a different level.

The bottom line: When looking for new hardware for your virtual hosts, be sure and choose servers that have one of these types of CPUs that are optimized for use with virtualization.