Creating a XEN Virtual Machine using command line tools

From WBITT's Cooker!

Jump to: navigation, search

Contents

Creating and installing a Xen Virtual Machine, using CLI tools only:

Scenario / Description:

  • Dell Laptop - 192.168.1.5 running Fedora 11 32 bit. (kworkbee)
  • Dell Desktop, Optiplex GX520- 192.168.1.100 running Centos 5.4 64 bit. (storage.example.com)
  • The laptop and Desktop systems are connected with each other on a physical network switch.
  • A Centos 53 32bit repository exists on dell laptop as http://192.168.1.5/centos53-32
  • A Kickstart file for CentOS exists as http://192.168.1.5/centos53-32/kickstart.cfg


The kickstart file looks like the following:

# Kickstart file for CENTOS-32bit installations on Virtual Machines
install
url --url http://192.168.1.5/centos53-32
lang en_US.UTF-8
keyboard us
network --device eth0 --bootproto dhcp --hostname vserver.example.com
rootpw --iscrypted $1$VQPyk3Ev$JePfY50WaA.aBhKT3xsBq.
firewall --disabled
authconfig --enableshadow --enablemd5
selinux --disabled
timezone Asia/Riyadh
bootloader --location=mbr --driveorder=sda
zerombr yes
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
clearpart --all --initlabel
part / --fstype ext3 --size=1000 --grow
part swap --size=512
reboot

%packages
@base

Xen stores it's disk files in /var/lib/xen/images. Create a 4GB disk for the new virtual machine, on physical / host machine:

[root@storage images]# dd if=/dev/zero of=/var/lib/xen/images/vserver.img bs=1M count=4000 
4000+0 records in                                                                          
4000+0 records out                                                                         
4194304000 bytes (4.2 GB) copied, 64.5974 seconds, 64.9 MB/s 


Now install the VM:

[root@storage ~]# virt-install -p -n vserver -r 512 -f /var/lib/xen/images/vserver.img  \ 
    -l http://192.168.1.5/centos53-32 -x ks=http://192.168.1.5/centos53-32/kickstart.cfg -w bridge:xenbr0 --vcpus=1 

Starting install...
Retrieving file vmlinuz...                                                                 | 2.1 MB     00:00     
Retrieving file initrd.img...                                                              | 6.0 MB     00:00     
Creating domain...                                                                         |    0 B     00:01     
Connected to domain vserver                                                                                       
Escape character is ^]                                                                                            
Linux version 2.6.18-128.el5xen (mockbuild@builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Wed Jan 21 11:55:02 EST 2009                                                                                
BIOS-provided physical RAM map:                                                                                   
 Xen: 0000000000000000 - 0000000020800000 (usable)                                                                
0MB HIGHMEM available.                                                                                            
520MB LOWMEM available.                                                                                           
NX (Execute Disable) protection: active                                                                           
ACPI in unprivileged domain disabled                                                                              
Built 1 zonelists.  Total pages: 133120                                                                           
Kernel command line:  method=http://192.168.1.5/centos53-32 ks=http://192.168.1.5/centos53-32/kickstart.cfg       
Enabling fast FPU save and restore... done.                                                                       
Enabling unmasked SIMD FPU exception support... done.                                                             
Initializing CPU#0                                                                                                
CPU 0 irqstacks, hard=c0744000 soft=c0724000                                                                      
PID hash table entries: 4096 (order: 12, 16384 bytes)                                                             
Xen reported: 2992.600 MHz processor.                                                                             
Console: colour dummy device 80x25                                                                                
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)                                                  
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)                                                    
Software IO TLB disabled                                                                                          
vmalloc area: e1000000-f4ffe000, maxmem 2d7fe000                                                                  
Memory: 505600k/532480k available (2124k kernel code, 18420k reserved, 877k data, 176k init, 0k highmem)          
Checking if this processor honours the WP bit even in supervisor mode... Ok.                                      
Calibrating delay using timer specific routine.. 7485.62 BogoMIPS (lpj=14971254)                                  
Security Framework v1.0.0 initialized                                                                             
SELinux:  Initializing.                                                                                           
selinux_register_security:  Registering secondary module capability                                               
Capability LSM initialized as secondary                                                                           
Mount-cache hash table entries: 512                                                                               
CPU: Trace cache: 12K uops, L1 D cache: 16K                                                                       
CPU: L2 cache: 2048K                                                                                              
Checking 'hlt' instruction... OK.                                                                                 
SMP alternatives: switching to UP code                                                                            
Freeing SMP alternatives: 13k freed                                                                               
Brought up 1 CPUs                                                                                                 
checking if image is initramfs... it is                                                                           
Freeing initrd memory: 7888k freed                                                                                
Grant table initialized                                                                                           
NET: Registered protocol family 16                                                                                
ACPI Exception (utmutex-0262): AE_BAD_PARAMETER, Thread C06F2AA0 could not acquire Mutex [2] [20060707]           
No dock devices found.                                                                                            
ACPI Exception (utmutex-0262): AE_BAD_PARAMETER, Thread C06F2AA0 could not acquire Mutex [2] [20060707]           
Brought up 1 CPUs                                                                                                 
PCI: setting up Xen PCI frontend stub                                                                             
ACPI: Interpreter disabled.                                                                                       
Linux Plug and Play Support v0.97 (c) Adam Belay                                                                  
pnp: PnP ACPI: disabled                                                                                           
xen_mem: Initialising balloon driver.                                                                             
usbcore: registered new driver usbfs                                                                              
usbcore: registered new driver hub                                                                                
PCI: System does not support PCI                                                                                  
PCI: System does not support PCI                                                                                  
NetLabel: Initializing                                                                                            
NetLabel:  domain hash size = 128                                                                                 
NetLabel:  protocols = UNLABELED CIPSOv4                                                                          
NetLabel:  unlabeled traffic allowed by default                                                                   
NET: Registered protocol family 2                                                                                 
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)                                                 
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)                                              
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)                                                       
TCP: Hash tables configured (established 131072 bind 65536)                                                       
TCP reno registered                                                                                               
audit: initializing netlink socket (disabled)                                                                     
type=2000 audit(1258692213.587:1): initialized                                                                    
VFS: Disk quotas dquot_6.5.1                                                                                      
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)                                                        
Initializing Cryptographic API                                                                                    
alg: No test for crc32c (crc32c-generic)                                                                          
ksign: Installing public key data                                                                                 
Loading keyring                                                                                                   
- Added public key 4F03F897CD3DCED2                                                                               
- User ID: CentOS (Kernel Module GPG key)                                                                         
io scheduler noop registered                                                                                      
io scheduler anticipatory registered                                                                              
io scheduler deadline registered                                                                                  
io scheduler cfq registered (default)                                                                             
pci_hotplug: PCI Hot Plug PCI Core version: 0.5                                                                   
rtc: IRQ 8 is not free.                                                                                           
Non-volatile memory driver v1.2                                                                                   
Linux agpgart interface v0.101 (c) Dave Jones                                                                     
RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize                                            
Xen virtual console successfully installed as xvc0                                                                
Linux version 2.6.18-128.el5xen (mockbuild@builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Wed Jan 21 11:55:02 EST 2009                                                                                
BIOS-provided physical RAM map:                                                                                   
 Xen: 0000000000000000 - 0000000020800000 (usable)                                                                
0MB HIGHMEM available.                                                                                            
520MB LOWMEM available.                                                                                           
NX (Execute Disable) protection: active                                                                           
ACPI in unprivileged domain disabled                                                                              
Built 1 zonelists.  Total pages: 133120                                                                           
Kernel command line:  method=http://192.168.1.5/centos53-32 ks=http://192.168.1.5/centos53-32/kickstart.cfg       
Enabling fast FPU save and restore... done.                                                                       
Enabling unmasked SIMD FPU exception support... done.                                                             
Initializing CPU#0                                                                                                
CPU 0 irqstacks, hard=c0744000 soft=c0724000                                                                      
PID hash table entries: 4096 (order: 12, 16384 bytes)                                                             
Xen reported: 2992.600 MHz processor.                                                                             
Console: colour dummy device 80x25                                                                                
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)                                                  
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)                                                    
Software IO TLB disabled                                                                                          
vmalloc area: e1000000-f4ffe000, maxmem 2d7fe000                                                                  
Memory: 505600k/532480k available (2124k kernel code, 18420k reserved, 877k data, 176k init, 0k highmem)          
Checking if this processor honours the WP bit even in supervisor mode... Ok.                                      
Calibrating delay using timer specific routine.. 7485.62 BogoMIPS (lpj=14971254)                                  
Security Framework v1.0.0 initialized                                                                             
SELinux:  Initializing.                                                                                           
selinux_register_security:  Registering secondary module capability                                               
Capability LSM initialized as secondary                                                                           
Mount-cache hash table entries: 512                                                                               
CPU: Trace cache: 12K uops, L1 D cache: 16K                                                                       
CPU: L2 cache: 2048K                                                                                              
Checking 'hlt' instruction... OK.                                                                                 
SMP alternatives: switching to UP code                                                                            
Freeing SMP alternatives: 13k freed                                                                               
Brought up 1 CPUs                                                                                                 
checking if image is initramfs... it is                                                                           
Freeing initrd memory: 7888k freed                                                                                
Grant table initialized                                                                                           
NET: Registered protocol family 16                                                                                
ACPI Exception (utmutex-0262): AE_BAD_PARAMETER, Thread C06F2AA0 could not acquire Mutex [2] [20060707]           
No dock devices found.                                                                                            
ACPI Exception (utmutex-0262): AE_BAD_PARAMETER, Thread C06F2AA0 could not acquire Mutex [2] [20060707]           
Brought up 1 CPUs                                                                                                 
PCI: setting up Xen PCI frontend stub                                                                             
ACPI: Interpreter disabled.                                                                                       
Linux Plug and Play Support v0.97 (c) Adam Belay                                                                  
pnp: PnP ACPI: disabled                                                                                           
xen_mem: Initialising balloon driver.                                                                             
usbcore: registered new driver usbfs                                                                              
usbcore: registered new driver hub                                                                                
PCI: System does not support PCI                                                                                  
PCI: System does not support PCI                                                                                  
NetLabel: Initializing                                                                                            
NetLabel:  domain hash size = 128                                                                                 
NetLabel:  protocols = UNLABELED CIPSOv4                                                                          
NetLabel:  unlabeled traffic allowed by default                                                                   
NET: Registered protocol family 2                                                                                 
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)                                                 
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)                                              
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)                                                       
TCP: Hash tables configured (established 131072 bind 65536)                                                       
TCP reno registered                                                                                               
audit: initializing netlink socket (disabled)                                                                     
type=2000 audit(1258692213.587:1): initialized                                                                    
VFS: Disk quotas dquot_6.5.1                                                                                      
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)                                                        
Initializing Cryptographic API                                                                                    
alg: No test for crc32c (crc32c-generic)                                                                          
ksign: Installing public key data                                                                                 
Loading keyring                                                                                                   
- Added public key 4F03F897CD3DCED2                                                                               
- User ID: CentOS (Kernel Module GPG key)                                                                         
io scheduler noop registered                                                                                      
io scheduler anticipatory registered                                                                              
io scheduler deadline registered                                                                                  
io scheduler cfq registered (default)                                                                             
pci_hotplug: PCI Hot Plug PCI Core version: 0.5                                                                   
rtc: IRQ 8 is not free.                                                                                           
Non-volatile memory driver v1.2                                                                                   
Linux agpgart interface v0.101 (c) Dave Jones                                                                     
RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize                                            
Xen virtual console successfully installed as xvc0                                                                
Event-channel device installed.                                                                                   
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2                                                          
ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx                                       
ide-floppy driver 0.99.newide                                                                                     
usbcore: registered new driver hiddev                                                                             
Welcome to CentOS                                                               

The system got an IP from the physical network and gave me a display right on the ssh shell !


Here is the log of httpd from my laptop computer:

192.168.1.100 - - [19/Nov/2009:19:41:56 +0300] "HEAD /centos53-32/ HTTP/1.1" 200 - "-" "Python-urllib/2.4"
192.168.1.100 - - [19/Nov/2009:19:41:56 +0300] "HEAD /centos53-32/.treeinfo HTTP/1.1" 404 - "-" "Python-urllib/2.4"                                                                                                                 
192.168.1.100 - - [19/Nov/2009:19:41:56 +0300] "HEAD /centos53-32/Fedora HTTP/1.1" 404 - "-" "Python-urllib/2.4"  
192.168.1.100 - - [19/Nov/2009:19:41:56 +0300] "HEAD /centos53-32/Server HTTP/1.1" 404 - "-" "Python-urllib/2.4"  
192.168.1.100 - - [19/Nov/2009:19:41:56 +0300] "HEAD /centos53-32/Client HTTP/1.1" 404 - "-" "Python-urllib/2.4"  
192.168.1.100 - - [19/Nov/2009:19:41:56 +0300] "HEAD /centos53-32/RedHat HTTP/1.1" 404 - "-" "Python-urllib/2.4"  
192.168.1.100 - - [19/Nov/2009:19:41:56 +0300] "HEAD /centos53-32/CentOS HTTP/1.1" 301 - "-" "Python-urllib/2.4"  
192.168.1.100 - - [19/Nov/2009:19:41:56 +0300] "GET /centos53-32/CentOS/ HTTP/1.1" 200 32117 "-" "Python-urllib/2.4"                                                                                                                
192.168.1.100 - - [19/Nov/2009:19:41:57 +0300] "HEAD /centos53-32/images/xen/vmlinuz HTTP/1.1" 200 - "-" "Python-urllib/2.4"                                                                                                        
192.168.1.100 - - [19/Nov/2009:19:41:57 +0300] "HEAD /centos53-32/images/xen/initrd.img HTTP/1.1" 200 - "-" "Python-urllib/2.4"                                                                                                     
192.168.1.100 - - [19/Nov/2009:19:41:57 +0300] "GET /centos53-32/images/xen/vmlinuz HTTP/1.1" 200 2188434 "-" "urlgrabber/3.1.0"                                                                                                    
192.168.1.100 - - [19/Nov/2009:19:41:57 +0300] "GET /centos53-32/images/xen/initrd.img HTTP/1.1" 200 6334185 "-" "urlgrabber/3.1.0"                                                                                                 
192.168.1.6 - - [19/Nov/2009:19:42:16 +0300] "GET /centos53-32/kickstart.cfg HTTP/1.0" 200 734 "-" "anaconda/11.1.2.168"                                                                                                            
192.168.1.6 - - [19/Nov/2009:19:42:16 +0300] "GET /centos53-32/images/updates.img HTTP/1.0" 404 305 "-" "anaconda/11.1.2.168"                                                                                                       
192.168.1.6 - - [19/Nov/2009:19:42:16 +0300] "GET /centos53-32/disc1/images/updates.img HTTP/1.0" 404 311 "-" "anaconda/11.1.2.168" 

As you can see, first the host machine (dell desktop) requested few things over httpd. And later the new virtual machine, when started the actual installation, started retrieving the packages. After the installation is completed, I see the following:

                                                                                
sending termination signals...done                                              
sending kill signals...done                                                     
disabling swap...                                                               
        /tmp/xvda2                                                              
unmounting filesystems...                                                       
        /mnt/runtime done                                                       
        disabling /dev/loop0                                                    
        /proc done                                                              
        /dev/pts done                                                           
        /sys done                                                               
        /tmp/ramfs done                                                         
        /selinux done                                                           
        /mnt/sysimage/sys done                                                  
        /mnt/sysimage/proc done                                                 
        /mnt/sysimage/selinux done                                              
        /mnt/sysimage/dev done                                                  
        /mnt/sysimage done                                                      
rebooting system                                                                
Restarting system.                                                              

Guest installation complete... restarting guest.
Connected to domain vserver                     
Escape character is ^]                          
Linux version 2.6.18-128.el5xen (mockbuild@builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Wed Jan 21 11:55:02 EST 2009                                                                                
BIOS-provided physical RAM map:                                                                                   
 Xen: 0000000000000000 - 0000000020800000 (usable)                                                                
0MB HIGHMEM available.                                                                                            
520MB LOWMEM available.             


I see the guest rebooting , above. You will see the guest OS login prompt as :

CentOS release 5.3 (Final)
Kernel 2.6.18-128.el5xen on an i686

vserver.example.com login:

Press Ctrl+] to exit from guest OS console view.

Ctrl+]
[root@storage ~]#

virt-install also creates the config file for this server, in /etc/xen:

[root@storage ~]# cat /etc/xen/vserver
name = "vserver"
uuid = "1a979429-92c1-ede2-1e5b-77f2596444b2"
maxmem = 512
memory = 512
vcpus = 1
bootloader = "/usr/bin/pygrub"
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "restart"
disk = [ "tap:aio:/var/lib/xen/images/vserver.img,xvda,w" ]
vif = [ "mac=00:16:36:72:28:ee,bridge=xenbr0,script=vif-bridge" ]
[root@storage ~]#


Let's connect to our new Virtual Server :

[kamran@kworkbee Documents]$ ssh root@192.168.1.6
The authenticity of host '192.168.1.6 (192.168.1.6)' can't be established.
RSA key fingerprint is 55:7e:84:1b:30:e3:77:98:4f:f6:8c:54:03:19:1d:56.   
Are you sure you want to continue connecting (yes/no)? yes                
Warning: Permanently added '192.168.1.6' (RSA) to the list of known hosts.
root@192.168.1.6's password:                                              
Last login: Fri Nov 20 07:48:35 2009                                      
[root@vserver ~]#

You would notice that the grub.conf file of the new virtual sevrer / guest system contains a special kernel boot option "console=xvc0"

[root@vserver ~]# cat /etc/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You do not have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /, eg.
#          root (hd0,0)
#          kernel /boot/vmlinuz-version ro root=/dev/xvda1
#          initrd /boot/initrd-version.img
#boot=/dev/xvda
default=0
timeout=5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.18-128.el5xen)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.18-128.el5xen ro root=LABEL=/ console=xvc0
        initrd /boot/initrd-2.6.18-128.el5xen.img
[root@vserver ~]#


Disk status is:

[root@vserver ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            3.3G  844M  2.3G  27% /
tmpfs                 257M     0  257M   0% /dev/shm
[root@vserver ~]#

[root@localhost ~]# fdisk -l

Disk /dev/xvda: 4194 MB, 4194304000 bytes
255 heads, 63 sectors/track, 509 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *           1         443     3558366   83  Linux
/dev/xvda2             444         508      522112+  82  Linux swap / Solaris
[root@localhost ~]#


The processor of the new virtual system is as follows. Notice the existance of "lm" flag, indicating that this vhost has 64 bit processor. Also remember that I assigned only one CPU for this server.

[root@vserver ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 6
model name      : Intel(R) Pentium(R) D CPU 3.00GHz
stepping        : 4
cpu MHz         : 2992.598
cache size      : 2048 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 6
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc up pni monitor ds_cpl vmx est cid cx16 xtpr lahf_lm
bogomips        : 7485.33

[root@vserver ~]#



[root@vserver ~]# cat /proc/meminfo
MemTotal:       524464 kB
MemFree:        353380 kB
Buffers:         11220 kB
Cached:         119108 kB
SwapCached:          0 kB
...
...


Just for information, the processor of the host / physical server is :-

[root@storage ~]# cat /proc/cpuinfo 
processor       : 0                 
vendor_id       : GenuineIntel      
cpu family      : 15                
model           : 6                 
model name      :               Intel(R) Pentium(R) D CPU 3.00GHz
stepping        : 4                                              
cpu MHz         : 2992.748                                       
cache size      : 2048 KB                                        
physical id     : 0                                              
siblings        : 1                                              
core id         : 0                                              
cpu cores       : 1                                              
fpu             : yes                                            
fpu_exception   : yes                                            
cpuid level     : 6                                              
wp              : yes                                            
flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est cid cx16 xtpr lahf_lm                                      
bogomips        : 7486.49                                                                                         
clflush size    : 64                                                                                              
cache_alignment : 128                                                                                             
address sizes   : 36 bits physical, 48 bits virtual                                                               
power management:                                                                                                 

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 15
model           : 6
model name      :               Intel(R) Pentium(R) D CPU 3.00GHz
stepping        : 4
cpu MHz         : 2992.748
cache size      : 2048 KB
physical id     : 1
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 6
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est cid cx16 xtpr lahf_lm
bogomips        : 7486.49
clflush size    : 64
cache_alignment : 128
address sizes   : 36 bits physical, 48 bits virtual
power management:

[root@storage ~]#

How about networking now?

Let's see the host server networking:

[root@storage ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:13:72:81:84:5B  
          inet addr:192.168.1.100  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::213:72ff:fe81:845b/64 Scope:Link              
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1              
          RX packets:130489 errors:0 dropped:0 overruns:0 frame:0         
          TX packets:9222 errors:0 dropped:0 overruns:0 carrier:0         
          collisions:0 txqueuelen:0                                       
          RX bytes:127795444 (121.8 MiB)  TX bytes:1193597 (1.1 MiB)      

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host     
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1405 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1405 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0                              
          RX bytes:2218688 (2.1 MiB)  TX bytes:2218688 (2.1 MiB) 

peth0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1     
          RX packets:265095 errors:0 dropped:0 overruns:0 frame:0
          TX packets:156402 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000                             
          RX bytes:385286022 (367.4 MiB)  TX bytes:11712481 (11.1 MiB)
          Interrupt:16 Memory:fe8f0000-fe900000                       

vif0.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1     
          RX packets:9249 errors:0 dropped:0 overruns:0 frame:0
          TX packets:130507 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0                                
          RX bytes:1197923 (1.1 MiB)  TX bytes:127796680 (121.8 MiB)

vif2.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:495 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1900 errors:0 dropped:71 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:66030 (64.4 KiB)  TX bytes:221797 (216.5 KiB)

virbr0    Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:468 (468.0 b)

xenbr0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:1237 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:158596 (154.8 KiB)  TX bytes:0 (0.0 b)

[root@storage ~]#


Whereas the networking on the virtual server is :

[root@vserver ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:36:72:28:EE
          inet addr:192.168.1.6  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::216:36ff:fe72:28ee/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1921 errors:0 dropped:0 overruns:0 frame:0
          TX packets:504 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:223597 (218.3 KiB)  TX bytes:73986 (72.2 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:560 (560.0 b)  TX bytes:560 (560.0 b)

[root@vserver ~]#


Shutting down and deleting a guest domain

[root@storage ~]# virsh 
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit               

virsh # 

virsh # list
 Id Name                 State
----------------------------------
  0 Domain-0             running
  2 vserver              idle

virsh # shutdown vserver
Domain vserver is being shutdown


virsh # list
 Id Name                 State
----------------------------------
  0 Domain-0             running

virsh # quit


[root@storage ~]# rm /etc/xen/vserver


To get a console of a virtual machine:

[root@storage ~]# virsh console vserver
Connected to domain vserver
Escape character is ^]

[root@localhost ~]#

OR

[root@storage ~]# virsh console vserver
Connected to domain vserver
Escape character is ^]

CentOS release 5.3 (Final)
Kernel 2.6.18-128.el5xen on an x86_64

localhost.localdomain login:
-b BRIDGE , --bridge=BRIDGE
    Bridge device to connect the guest NIC to. This parameter is deprecated in favour of the "--network" parameter. 
-w NETWORK , --network=NETWORK
    Connect the guest to the host network. The value for "NETWORK" can take one of 3 formats: 
    bridge:BRIDGE 
    Connect to a bridge device in the host called "BRIDGE". Use this option if the host has static networking config & the guest requires full outbound and inbound connectivity to/from the LAN . Also use this if live migration will be used with this guest. 
    network:NAME 
    Connect to a virtual network in the host called "NAME". Virtual networks can be listed, created, deleted using the "virsh" command line tool. In an unmodified install of "libvirt" there is usually a virtual network with a name of "default". Use a virtual network if the host has dynamic networking (eg NetworkManager), or using wireless. The guest will be NATed to the LAN by whichever connection is active. 
    user 
    Connect to the LAN using SLIRP . Only use this if running a QEMU guest as an unprivileged user. This provides a very limited form of NAT . 
    If this option is omitted a single NIC will be created in the guest. If there is a bridge device in the host with a physical interface enslaved, that will be used for connectivity. Failing that, the virtual network called "default" will be used. This option can be specified multiple times to setup more than one NIC . 
--arch=ARCH
    Request a non-native CPU architecture for the guest virtual machine. The option is only currently available with QEMU guests, and will not enable use of acceleration. If omitted, the host CPU architecture will be used in the guest. 


If something goes wrong during installation, you can shutdown the domain through virsh and then start over again with your steps. Also remove the virtual machine config file, otherwise you will get an error during virt-install, such as:

ERROR    Guest name 'vserver' is already in use.
ERROR    A name is required for the virtual machine. (use --prompt to run interactively)

So delete the domain config file. Make sure the domain is not active before you delete the config file.

[root@storage ~]# rm /etc/xen/vserver
rm: remove regular file `/etc/xen/vserver'? y
[root@storage ~]#


Scenario 2

  • Create a virtual machine on the physical host. It should be on a virtual network inside the physical host. Not connected to the physical network directly.
  • The only thing which will change is the -w switch during virt-install . I will also use 64bit OS this time, instead of 32 bit, just for fun!
  • That is, -w bridge:xenbr0 will change to -w network:default
virt-install -p -n vserver -r 512 -f /var/lib/xen/images/vserver.img -l http://192.168.1.5/centos53-64 -x ks=http://192.168.1.5/centos53-64/kickstart.cfg -w network:default --vcpus=2

I see success :

... 
... 
Starting Avahi daemon... [  OK  ]
Starting HAL daemon: [  OK  ]
Starting smartd: [  OK  ]

CentOS release 5.3 (Final)
Kernel 2.6.18-128.el5xen on an x86_64

localhost.localdomain login: root
Password:
[root@localhost ~]#

The deafult is the virtual network within the physical host, as you can see in the scenario 2 setup:

[root@localhost ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:36:24:82:12
          inet addr:192.168.122.93  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::216:36ff:fe24:8212/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:48 errors:0 dropped:0 overruns:0 frame:0
          TX packets:44 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3101 (3.0 KiB)  TX bytes:8727 (8.5 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:560 (560.0 b)  TX bytes:560 (560.0 b)

[root@localhost ~]#


The physical host's networking looks like this:

[root@storage ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:13:72:81:84:5B  
          inet addr:192.168.1.100  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::213:72ff:fe81:845b/64 Scope:Link              
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1              
          RX packets:551455 errors:0 dropped:0 overruns:0 frame:0         
          TX packets:189174 errors:0 dropped:0 overruns:0 carrier:0       
          collisions:0 txqueuelen:0                                       
          RX bytes:683216698 (651.5 MiB)  TX bytes:14053664 (13.4 MiB)    

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host     
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1406 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1406 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0                              
          RX bytes:2218764 (2.1 MiB)  TX bytes:2218764 (2.1 MiB) 

peth0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1     
          RX packets:726851 errors:0 dropped:0 overruns:0 frame:0
          TX packets:418332 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000                             
          RX bytes:1058738429 (1009.6 MiB)  TX bytes:31043400 (29.6 MiB)
          Interrupt:16 Memory:fe8f0000-fe900000                         

vif0.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1     
          RX packets:189185 errors:0 dropped:0 overruns:0 frame:0
          TX packets:551466 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0                                
          RX bytes:14056246 (13.4 MiB)  TX bytes:683217424 (651.5 MiB)

vif5.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:44 errors:0 dropped:0 overruns:0 frame:0
          TX packets:79 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:8111 (7.9 KiB)  TX bytes:4713 (4.6 KiB)

virbr0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:164260 errors:0 dropped:0 overruns:0 frame:0
          TX packets:280231 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:8712571 (8.3 MiB)  TX bytes:417362823 (398.0 MiB)

xenbr0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:3725 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1107541 (1.0 MiB)  TX bytes:0 (0.0 b)

[root@storage ~]#

Checking the same from the virsh interface:

virsh # net-list
Name                 State      Autostart
-----------------------------------------
default              active     yes

virsh #

virsh # list
 Id Name                 State
----------------------------------
  0 Domain-0             running
  5 vserver              idle


virsh # vcpuinfo 5
VCPU:           0
CPU:            1
State:          idle
CPU time:       6.1s
CPU Affinity:   yy

VCPU:           1
CPU:            1
State:          idle
CPU time:       4.4s
CPU Affinity:   yy

virsh #

OR

virsh # vcpuinfo vserver
VCPU:           0
CPU:            1
State:          idle
CPU time:       6.1s
CPU Affinity:   yy

VCPU:           1
CPU:            1
State:          idle
CPU time:       4.4s
CPU Affinity:   yy

virsh #


The config file of virtual domain for this scenario is like:

[root@storage ~]# cat /etc/xen/vserver
name = "vserver"
uuid = "b658744d-5737-c7d9-e912-1f73d1670313"
maxmem = 512
memory = 512
vcpus = 2
bootloader = "/usr/bin/pygrub"
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "restart"
disk = [ "tap:aio:/var/lib/xen/images/vserver.img,xvda,w" ]
vif = [ "mac=00:16:36:24:82:12,bridge=virbr0,script=vif-bridge" ]
[root@storage ~]#


virsh # dominfo vserver                                
Id:             5                                
Name:           vserver                          
UUID:           b658744d-5737-c7d9-e912-1f73d1670313
OS Type:        linux                               
State:          idle                                
CPU(s):         2                                   
CPU time:       10.9s                               
Max memory:     524288 kB                           
Used memory:    524124 kB                           
Autostart:      disable        

The vif5.0 on physical host, as checked with ifconfig command on physical host, is connected to network card on virtual host (domain 5).

virsh # domifstat vserver vif5.0
vif5.0 rx_bytes 56771
vif5.0 rx_packets 1178
vif5.0 rx_errs 0
vif5.0 rx_drop 0
vif5.0 tx_bytes 8139
vif5.0 tx_packets 45
vif5.0 tx_errs 0
vif5.0 tx_drop 0

Question: how do we know, which interface is used by which virtual host and what is it's IP ?

Check which nodes on the virtual network are alive.

[root@storage ~]# nmap -sP 192.168.122.0/24

Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2009-11-20 12:11 AST
Host 192.168.122.1 appears to be up.
Host 192.168.122.93 appears to be up.
MAC Address: 00:16:36:24:82:12 (Quanta Computer)
Nmap finished: 256 IP addresses (2 hosts up) scanned in 5.923 seconds
[root@storage ~]#
[root@storage ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.feffffffffff       yes             vif1.0
xenbr0          8000.feffffffffff       no              peth0
                                                        vif0.0
[root@storage ~]#


Setting up a web service on the virtual machine, which is on a private virtual network inside physical host. Accessing it from another physical host from outside.

  • Physical host providing VM services / xen [Desktop]: 192.168.1.100
  • VM IP : 192.168.122.93
  • Client physical host [laptop] : 192.168.1.5
Personal tools