Virtualization
From WBITT's Cooker!
A little about the Author:
Muhammad Kamran Azeem [ CISSP, RHCE, OCP (DBA) ]. Working on High Performance Computing Clusters at Saudi Aramco. More at http://wbitt.com
Contents |
Virtualization
- What is Virtualization?
- Commonly known virtualization technologies?
- Advantages and Disadvantages of Virtualization?
- Types of Virtualization (Para / Full, etc)
- Types of Hyper-visors
- Technologies we will cover: XEN, KVM
- Note: Most of the material was obtained from Wikipedia http://en.wikipedia.org/wiki/
What is Virtualization?
Virtualization is a term that refers to the abstraction of computer resources. In simpler words, the mechanism to run multiple instances/copies of various operating systems inside a base operating system, mainly to utilize under-used resources on the physical host, where base operating system is running.
History of Virtualization
- The IBM System/360 Model 67 (S/360-67) was a mainframe, and first shipped in July 1966. It included features to facilitate time-sharing applications, notably virtual memory hardware and 32-bit addressing.
- CP/CMS was the first fully-virtualized virtual machine operating system, running on IBM System/360 Model 67, and evolved from the ground-breaking research system CP-40.
- The S/360-67 included various hardware and microcode features that enabled full virtualization of the raw S/360 hardware. The full-virtualization concept was pioneered with CP-40 on custom hardware; its implementation on the S/360-67 made CP-67 possible.
- It is important to note that full hardware virtualization was not an original design goal for the S/360-67.
- Thus, in many respects, it can be said, that IBM's CP-67 and CP/CMS products anticipated (and heavily influenced) contemporary virtualization software, such as VMware Workstation, Xen, and Microsoft Virtual PC.
- The IBM System/370 (S/370) was a model range of IBM mainframes announced on June 30, 1970 as the successors to the System/360 family.
- Full virtualization was not quite possible with the x86 platform until the 2005-2006 addition of the AMD-V and Intel VT extensions.
- Many platform virtual machines for the x86 platform came very close, and claimed full virtualization even prior to the AMD-V and Intel VT additions. e.g. Parallels Workstation, VMware Workstation, VMware Server (formerly GSX Server), VirtualBox, etc.
VMware
- VMware was founded in 1998 and delivered its first product, VMware Workstation, in 1999.
- VMware and similar virtualization software for the x86 processor family must employ binary translation techniques to trap and virtualize the execution of certain instructions. These techniques incur some performance overhead as compared to a VM running on a natively virtualizable architecture.
- VMware is closed source.
Xen
- Xen originated as a research project (XenoServer) at the University of Cambridge, led by Ian Pratt, who later founded XenSource, Inc.
- XenSource supports the development of the open source project and also sells enterprise versions of the software.
- Details about Xen's design are in the 2003 research paper: Xen and the Art of Virtualization
- Xen is open source software.
- The first public release of Xen occurred in 2003.
- Citrix Systems acquired XenSource, Inc in October 2007 and subsequently renamed Xensource's products under the Citrix brand.
- Xen Management Consoles
- Xen Tools
- Ganeti
- Perl-based MLN
- Web-based HyperVM and FluidVM, Cloudmin
- GUI applications Convirture (formerly XenMan) and Red Hat's Virtual Machine Manager, virt-manager.
- Novell's PlateSpin Orchestrate also manages Xen VMs in SUSE Linux Enterprise Server.
- Xen supported architectures are:
- 32-bit x86 with PAE support
- Intel 64/AMD64
- Intel Itanium 2
- Xen's Full-Virtualization additionally requires availability of Intel VT-x or AMD-V technology within the processor.
KVM
- KVM is open source software.
- KVM ( Kernel-based Virtual Machine) was developed by Qumranet, Inc.
- On September 4, 2008, Qumranet was acquired by Red Hat, Inc.
- KVM is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMD-V).
- Using KVM, one can run multiple VMs running unmodified Linux or Windows images.
- Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter, etc.
- The kernel component of KVM is included in mainline Linux, as of 2.6.20.
- KVM management tools: ovirt, Virtual Machine Manager, etc.
Other examples of x86 virtualization software
- Microsoft's Virtual PC, Hyper-V, and Microsoft Virtual Server.
- Open-source solutions: QEMU, Kernel-based Virtual Machine (KVM) and VirtualBox.
- Research systems: Denali, L4, and Xen.
- The following software conditionally makes use of the support offered by AMD-V and/or Intel VT:
- KVM, VirtualBox, Xen, VMware ESX Server (also known as vSphere). Microsoft Hyper-V, Microsoft Virtual Server (also branded as Microsoft Virtual PC or Windows Virtual PC), Oracle VM (uses Xen). Parallels Workstation, Parallels Server, Sun xVM, Virtual Iron, VMware Workstation, VMware Fusion, VMware Server.
Virtualization terminology
- Hyper-Visor or Virtual Machine Monitor (VMM) : It is the software which manages and supports the virtualization environment. It runs the virtual machines and isolates them from real hardware. There are three types of Hyper-Visors.
- Type 1 Hyper-visor: A hyper-visor running on bare metal hardware, e.g. Linux KVM, IBM z/VM, VMware ESX, etc
- Type 2 Hyper-visor: Virtualization software that runs on the host OS. e.g. VMware workstation, VMware server (formerly known as GSX server), Parallels Desktop, Microsoft Virtual Server, etc.
- Hybrid Hyper-visor: Runs directly on bare metal like Type 1, but depends heavily on drivers and support from one of its (privileged) virtual machines to function properly. e.g. Xen. Dom-0 is the special VM, which is needed by kernel-xen.
- Emulator: Emulator is a software which emulates all pieces of hardware for it's VM. e.g. VMware, Qemu, etc.
- Shared Kernel: Used in chrooted / jailed virtual environments. All machines share the same kernel, and most of the libraries. Only some parts of the OS are (so called) "virtualized", or made available to the VM through separate directories.
- Domain: Any virtual machine running on hyper-visor.
- Domain-0 / Privileged Domain: A virtual machine having privileged access to the hyper-visor. It manages the hypervisor and the other VMs. This domain is always started first by the hyper-visor, on system boot. Also referred to as Management Domain or Management Console. Dom-0 can be used in "Thick" or "Thin" model. Thick model means that a lot of software is present to assist virtual machine management. Such as laptops, desktops, etc, used for development and testing. Thin model means that Dom-0 is kept as thin as possible by providing just the bare minimum software components to the hyper-visor to run the virtual machines properly. This results in lesser resource utilization by the Dom-0, and providing more resources to the guest domains. Used in production environments, on production servers, etc.
- Domain-U / Guest Domains / User Domains: VM created by Dom-0. Sometimes simply known as Guest, or Dom-U.
- PAE: Physical Address Extension, is a feature first implemented in the Intel Pentium Pro to allow x86 processors to access more than 4 gigabytes of random access memory if the operating system supports it. It was extended by AMD to add a level to the page table hierarchy, to allow it to handle up to 52-bit physical addresses, add NX bit functionality, and make it the mandatory memory paging model in long mode.
- PAE is provided by Intel Pentium Pro (and above) CPUs - including all later Pentium-series processors except the 400 MHz bus versions of the Pentium M, as well as by other processors such as the AMD Athlon and later AMD processor models with similar or more advanced versions of the same architecture.
- Required to be present on 32-bit x86 CPU, if para-virtualization is to be used. (This means that you can most certainly use Xen for-para-virtualization on almost any hardware lying around in your office / home.)
- Intel VT-x
- Intel VT (Virtualization Technology) is the Intel's hardware assistance for processors running virtualization platforms.
- Intel VT includes a series of extensions for hardware virtualization. The Intel VT-x extensions, adds migration, priority and memory handling capabilities to a wide range of Intel processors. By comparison, the VT-d extensions add virtualization support to Intel chipsets that can assign specific I/O devices to specific virtual machines (VM)s, while the VT-c extensions bring better virtualization support to I/O devices such as network switches.
- AMD-V
- AMD-V (AMD Virtualization) is a set of hardware extensions for the x86 processor architecture. AMD designed the extensions to perform repetitive tasks normally performed by software and improve resource use and virtual machine (VM) performance.
- AMD-V technology was first announced in 2004 and added to AMD's Pacifica 64-bit x86 processor designs.
- By 2006, AMD's Athlon 64 X2 and Athlon 64 FX processors appeared with AMD-V technology, and today, the technology is available on Turion 64 X2, second- and third-generation Opteron, Phenom and Phenom II processors.
Processor capability identification tips:
On linux, you can check the /proc/cpuinfo file and see if the flags line has "vmx" (for Intel) or smx (for AMD) in it. If the following line results in some text, then your CPU (irrespective of being Intel or AMD), has Hardware-Assisted Full Virtualization support.
egrep ‘(vmx|svm)’ /proc/cpuinfo
If the command above, does not return any results, or just returns to the command prompt silently, then your processor does not support Hardware-Assisted Full Virtualization. However, it should be noted that sometimes, this feature is turned off in the BIOS. Therefore you should check your BIOS settings first, to verify that.
Also, if your CPU is an older model, and does not have Intel VT-x or AMD-V technologies, all hope is not lost. Check if your CPU provides PAE feature. If that is there, you can still use/create Para-Virtual virtual machines on this machine, using Xen. You can also use the emulation based full virtualization products such as Qemu, Bochs, Virtual Box, VMware workstation, etc. Here is how you can check for PAE for your CPU in Linux:
grep pae /proc/cpuinfo
Additional tip to check if your processor is 64 bit or not is to check for a flag "lm" or "long mode" in the cpu flags. If the command below returns some text, you have a 64 bit processor:
grep -w lm /proc/cpuinfo
Why use Virtualization?
- Consolidation
- Power, Rack/Desk/Floor Space, Hardware, HVAC, Wiring/cabling, etc.
- Efficient utilization of under-utilized resources
- CPU / memory, disks, bandwidth, etc.
- Support for applications only supporting older versions of some OS.
- Service / domain / role based compartmentalization
- e.g. mail server and web server on separate VMs.
- Fail-over and Load Balancing features.
- Development and Testing
- Developers can test code on test servers.
- Easy roll-backs.
- Replica of production server can be created as a VM, so patches, etc can be tested.
- Programs/Applications targeted to run on different OS / platforms can be tested. e.g a web application in need to be tested on firefox running on Linux and on Windows.
- Virus testing, Spam testing, password cracking, sniffing, DOS, etc can all be tested safely.
- Training
- Virtual labs can be setup with less resources
- Security training can be delivered without concerns of busting out in the production network.
- Each student can have his own (virtual) lab in his own PC/Laptop, in addition to the lab provided by the instructor.
- Virtual Appliances
- Appliances, such a a hardened mail server, can be created, which simply would need to be started as a VM in your host OS, etc. Same can be done to create fully functional web hosting servers. (I have deployed few web hosting servers, using this method! )
- Ease of machine migration in case of hardware failure. (e.g. No need to re-install / reconfigure your favourite mail server from scratch!)
- Legacy application support
- Legacy applications / programs , which do not support latest hardware or OS, etc, can be made to run on the OS they support, in a virtualized environment.
- Such applications benefit from the newer hardware, such as speed , and thus run faster.
- And someone said, less screw-drivers!
Why “not” Virtualization?
- Administration of VMs, when more than a few, is more complicated, and sensitive than more than a few physical servers. The different VM interfaces, such as VMware's Virtual Infrastructure Center, and KVM's ovirt, try to address this.
- For live migrations involving movement of a VM from one physical host to another, involves extra IPs, etc. Plus the shared storage, sometimes cluster file systems, etc.
- Various networking problems arise, such as firewalls, routing, switching, bridging etc.
- Some service providers (such as ServerBeach), does not support “bridged connections” from the rented server. This adds extra complexity in managing the physical host's firewall, routing tables, etc.
- Hardware needs to be more fault-tolerant, and relatively powerful, compared to single server/service requirements.
- Console access, block device access, recovery, system trouble-shooting, etc, are complex areas to handle.
Virtualization Technologies
Emulation-based Full Virtualization
- Slower than hardware-based full virtualization.
- Hyper-visor simulates the virtual machine in software, by analyzing all instructions and converting each one appropriately before it gets to the CPU.
- Dynamic translation is a technique used to improve performance. Which is, the hypervisor analyses the binary instructions just before they are run, allowing safe instructions to run unmodified, but converting sensitive instructions just before they execute. The converted code is also cached in memory, to speed up future (sensitive) instructions coming in for execution.
- Dynamic recompilation optimizes frequently reused sequences on the fly.
- Full vitualization with Dynamic Recompilation is the basic technique used by VMware, for it's initial / basic products, VMware Workstation, VMware Server , etc.
- Full emulation can also be used to simulate non-standard processor architectures, needed by different OS / applications, by converting all instructions.
- This method of simulating/emulating results in very slow VMs.
- QEMU, Bochs are example of non-native / non-standard processor emulators for/on Linux.
Native / Hardware-based / Hardware-assisted Full Virtualization
- Requires CPU based hardware acceleration. (Intel VT-x, AMD-V)
- Bare-metal look and feel. Access to HW is controlled through hyper-visor.
- Almost all code coming in from VM is run directly by the CPU of the physical host, without any changes, for efficiency.
- The hyper-visor only steps in when the code coming in from the VM uses sensitive instructions, that would interfere with the state of the hyper-visor itself, or the environment it is supported by.
- Such sensitive instructions must be intercepted by the hyper-visor and translated/replaced with safe equivalents before they are actually executed on the CPU of the physical host.
- To do this, all sensitive instructions in the CPU's Instruction Set Architecture (ISA), must be defined as privileged.
- Traditional x86 architecture's instruction set has about 17 instructions which are sensitive, but they are not defined as privileged, which are unable to trap such instructions coming from VM. The latest Intel Itanium-2 has three instructions which are sensitive, but are still not defined as privileged.
- Intel VT-x and AMD-V technologies were developed to overcome this problem on modern 32-bit and 64 bit x86 processors.
- In Linux, Xen hyper-visor "can use" these new CPU features. Whereas, KVM "needs/requires" these features in the CPU, for it (KVM hyper-visor) to work.
- Examples are: KVM, VMware ESX
- Un-modified Guest OS can be used as VM. e.g. Windows.
Para Virtualization / Cooperative Virtualization
- Works without the newly available CPU based hardware acceleration technologies, such as Intel VT-x and AMD-V.
- e.g. Xen.
- The "hyper-visor aware" code is integrated into the kernel of the operating systems running on the virtual machines. This results in a "modified kernel", commonly known as "kernel-xen" instead of simply "kernel". That is why you will see "kernel-xen-x.y" booting up when you power up your virtual machine OS. The base OS / Domain-0 already runs under kernel-xen. Generally, no other changes are required on the rest of the software on the virtual machines. Xen is the actual hyper-visor, which runs directly on the CPU of the physical host, with "full speed". In other words, the (modified) kernel of each virtual machine's OS actually runs on the hyper-visor, assuming the hyper-visor to be CPU itself. This happens to all the VMs. This eliminates the need to have a separate trapping / translation mechanism to be present in the hyper-visor.
- The above description implies that only modified Guest OS can be used as VM, which understand the hyper-visor. That means windows and family products cannot be run in Para-Virtualization environment. (One of the excellent books on Xen: "The book of Xen", mentions that, though Xen team did port Windows to Xen during the development process, no "released" version of Windoes can run under Xen in para-virtualized mode.)
- Only the hyper-visor has privileged access to the CPU, and is designed to be as small and limited as possible.
- The Xen hyper-visor interacts with the OS running under it's control, using very few well-defined interfaces, called hyper-calls. Xen has about 50 hyper-calls compared to about 300 for Linux!
- Hyper-calls are "asynchronous",so that the hyper-calls themselves don't block other processes or other OSes.
- The Xen paper, mentioned earlier (Xen and the art of virtualization), indicates performance degradation of less than 2 percent for standard work-load scenarios. And a degradation of between 10 and 20 percent for worst case scenarios!
- The base OS, which actually installs Xen hyper-visor on the physical host, is also referred to as "Privileged Domain" or "Domain-0" or "Dom-0". This privileged domain is in-turn used to manage the hypervisor. This privileged domain manages all other virtual machines created under Xen hyper-visor. These other virtual machines are referred to as "Guest Domains" or "User Domains" or "Dom-U". That means the OS of the privileged domain, also runs as a VM, under Xen hyper-visor, just like other virtual machines on the same physical host, "but", "with more privileges". Dom-0 has direct access to the hardware resources of the physical host.
- Advantages are :
- Works on older hardware, or on the hardware which does not have hardware-assisted full virtualization.
- Works much faster than Emulated or Hardware based virtualization technologies. Sometimes outperforming the actual bare metal performance!
- Performance can further be enhanced by presenting virtual block devices to the virtual machines, instead of real block devices. This means that special para-virtualized drivers need to be present in the OS running on the VM. The co-operation between the kernel and the hyper-visor can allow para-virtualized drivers to have much lower overhead than native drivers.
- Advantages are :
OS Virtualization
- OS Chroot environments.
- OpenVZ, Solaris Containers, FreeBSD jails, etc.
- Shared kernel is the single point of failure.
Application Virtualization
- Application creates a sandbox environment in browser, etc. e.g. JRE .
API-level Virtualization
- Virtualization provided to support single application.
- e.g. WINE is used to run Windows programs in Linux environment.
