CENTOS Server CD project
From WBITT's Cooker!
Contents |
Why Server CD?
Lately I thought of installing a XEN CENTOS 5.5 virtual machine on one of my servers out there. I needed the exploded tree of the ISO to be available. However, when I checked centos.org website, I noticed that they stopped distributing a "Single Server CD" after CENTOS 4. I could have downloaded a CENTOS 5.5 DVD image on the server but it was kind of un-desired / not practical for two reasons.
- The centos.org distributes DVD iso image through bit-torrent, and my server service provider does not allow P2P or torrent traffic.
- Even if I had one, I would have to download one big 4-5 GB file to my server. Whereas a few hundred MBs would do, if I want a text based "core" installation only.
So there arose an itch, and I decided to create a project to scratch that itch!
Objectives / Project goals
- To create a repository / collection of files, a so called "Server CD" install-tree, small enough to provide the "core" installation, plus yum and wget.
- To be able to create a boot-able /installable "install CD" out of the install-tree mentioned above. So that, when I (or anyone else) need to install a minimum OS on a bare-metal machine, he doesn't have to go through downloading a full DVD.
Creating the "core" install-tree, the hard way
The idea is simple. I have a CENTOS 5.4 DVD already downloaded at home (and office). To get the list of packages, I did a "core" install of CENTOS on a XEN virtual machine. (I used a kickstart file with @core mentioned in the "%packages" section. Analysing its "/root/install.log", I quickly figured out which packages got installed. (clever me!). So this is the list of packages I need from the "CentOS" directory of the install-tree. (It's about 443 packages). This resulted in an installation of about 950MB in size.
Here is how I used (install.log) to generate the list of required pacakges in my repository.
[root@xenlab minivm]# grep "Installing" install.log | awk '{print $2}'|sort > installed-rpms.tmp
Then I edited the resulting file to remove strange numbers before the package names, such as:
basesystem-8.0-5.1.1.el5.centos.noarch cracklib-dicts-2.8.9-3.3.x86_64 1:termcap-5.5-1.20060701.1.noarch <<<<------- Such as this one. What is this number ? nash-5.1.19.6-54.x86_64
This is good enough. However, I am too much irritated by seeing bluetooth, cups, x11, and similar packages being part of the core installation. So I went one step further, and started deleting packages using yum, in a polite fashion, from this VM. This way, I was able to delete some packages, which I consider un-necessary for an installation, where only a web-server is the only desired service to run, without any backend database. (Which was the case with me.). After this surgery, I was able to bring down the disk usage to about 750 MB. And of-course, fewer number of packages. (About 118 less rpms!). I can use that in the %post to delete the un-necessary RPMs!
# yum remove aspell aspell-en atk authconfig autofs avahi avahi-compat-libdns_sd \ bitstream-vera-fonts bluez-gnome bluez-libs bluez-utils cairo ccid coolkey \ cpuspeed crash cups cups-libs desktop-file-utils dhcpv6-client dnsmasq \ dos2unix dosfstools ecryptfs-utilsed eject fbset finger firstboot-tui \ fontconfig freetype GConf2 gpm gtk2 hicolor-icon-theme htmlview ifd-egate \ iptables-ipv6 irda-utils irqbalance jwhois krb5-workstation ksh libdrm \ libICE libjpeg libnotify libpng libSM libtiff libwnck libX11 libXaulibXcursor \ libXdmcp libXext libXfixes libXft libXi libXinerama libXrandr libXrender \ libXres libXt libXxf86vm mailcap man man-pages mdadm mesa-libGL microcode_ctl \ mkbootdisk mtools nano NetworkManager NetworkManager-glib newt \ notification-daemon ntsysv numactl ORBit2 pam_ccreds pam_krb5 pam_pkcs11 \ pam_smb pango paps pcmciautils pcsc-lite pcsc-lite-libs pinfo procmail rdate \ redhat-lsb redhat-menus rhpl rp-pppoe rsh sendmail setuptool slang sos \ specspo startup-notification syslinux system-config-network-tui \ system-config-securitylevel-tui tcpdump trousers unix2dos vconfig \ wireless-tools words wpa_supplicant xorg-x11-filesystem ypbind \ yp-tools yum-updatesd
Tip: I used the following commands to generate the long yum command above:
# grep Erased /var/log/yum.log | awk '{ print $5}' | sort | uniq | tr '\n' ' '
To create my minimal repository, I first created a copy of the original install tree skipping the CentOS directory.
[root@xenlab ~]# rsync -av --exclude CentOS/ \ /var/www/html/CentOS-5.4-x86_64/ /var/www/html/CentOS-5.4-x86_64-core/
Then I used the list of packages from the install.log file, and pulled the matching RPM files from the full install-tree into this "core" install-tree. I used the "installed-rpms.txt" file, I created a moment ago, above. The installed-rpms.txt file looks like this:
[root@xenlab ~]# cat /tmp/minivm/installed-rpms.tmp | tail which-2.16-7.x86_64 words-3.0-9.1.noarch xorg-x11-filesystem-7.1-2.fc6.noarch yp-tools-2.9-0.1.x86_64 yum-3.2.22-20.el5.centos.noarch yum-fastestmirror-1.1.16-13.el5.centos.noarch yum-metadata-parser-1.1.2-3.el5.centos.x86_64 zip-2.31-2.el5.x86_64 zlib-1.2.3-3.i386 zlib-1.2.3-3.x86_64 [root@xenlab ~]#
Create the CentOS directory in the "core" install tree:
[root@xenlab ~]# mkdir /var/www/html/CentOS-5.4-x86_64-core/CentOS
Then copy the selected files from the full install-tree into this "core" tree, using the installed-rpms.txt file.
[root@xenlab ~]# for file in $(cat /tmp/minivm/installed-rpms.tmp); do
cp -v /var/www/html/CentOS-5.4-x86_64/CentOS/"${file}.rpm" \
/var/www/html/CentOS-5.4-x86_64-core/CentOS-5.4-x86_64-core/CentOS/
done
Check the "core" CentOS directory:
[root@xenlab ~]# cd /var/www/html/CentOS-5.4-x86_64-core/CentOS/ [root@xenlab CentOS]# ls -l | wc -l 443 [root@xenlab CentOS]# du -sh . 273M . [root@xenlab CentOS]#
Lovely! As you can see only 443 RPMs and 273 MB occupied by the RPMs only!
Note: We need to (additionally) copy the normal kernel to the new location too, because someone might want to use this for installing a non-XEN machine, or for the bare-metal machine. In that case a normal kernel would be required.
[root@xenlab ~]# cp /var/www/html/CentOS-5.4-x86_64/CentOS/kernel-2.6.18-164.el5.x86_64.rpm \ /var/www/html/CentOS-5.4-x86_64-core/CentOS/ [root@xenlab ~]#
Notice the difference now:
[root@xenlab CentOS]# cd /var/www/html/CentOS-5.4-x86_64-core/CentOS/ [root@xenlab CentOS]# ls | wc -l 444 [root@xenlab CentOS]# du -sh . 291M .
The complete usage of the minimal install tree is :-
[root@xenlab ~]# du -sh /var/www/html/CentOS-5.4-x86_64-core 606M /var/www/html/CentOS-5.4-x86_64-core [root@xenlab ~]#
Note: In all the steps above, we DID NOT update/change the repository (meta) data, (using createrepo command). Nor did we change anything in the installer XML files. It works just like that.
When I was done with doing so, I created another XEN VM, and used this new (shrunk) install-tree to install the new VM to make sure it works. And it did! Here is the newly installed system:-
[root@localhost ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/xvda1 2.7G 986M 1.6G 39% / tmpfs 256M 0 256M 0% /dev/shm [root@localhost ~]# rpm -qa | wc -l 443 [root@localhost ~]#
Then I removed the 118 packages, which I found un-necessary (as shown in the yum remove command above). The new space usage of this VM is now:
[root@localhost ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/xvda1 2.7G 775M 1.8G 31% / tmpfs 256M 0 256M 0% /dev/shm [root@localhost ~]#
Now, to use this install tree on my server on the internet, I can do one of the following:
- Push the repository tarball to my XEN physical host/server on internet,
- Or, make a script to "wget" all the required packages from a CENTOS mirror, create a repo for the installer, and make that available through yum on that server.
The first method is kind of immediately do-able, but not "exciting". It definitely involves copying all the install tree to the XEN host up there. To do the second method, I created a kind of script, which can pull the packages from any install tree on the internet and populate the CentOS directory on my own install-tree. Good enough, but still not geeky enough.
What I want to do now, is to be able to parse comps.xml file, present in the repodata directory of any regular install tree, find out the packages needed for "@core" and then pull those packages from the regular install tree into my own install tree. (This would be geeky/cool.)
And when I can do that, I can then create an actual bootable ISO out of it using help from many posts on the internet (mentioned at the end of this document).
Creating the "core" install-tree from a http site
Here, I am using wget to get the basic structure of CentOS full repository to my local machine, "without" pulling any rpms.
Notes:
- In the example below, /tmp/centos-mini is the location where I am building the "core" install-tree.
- 192.168.0.1 is a web server, with complete DVD install-tree exported through web service.
- Interestingly CentOS directory is the "only" directory in any install tree, which contains RPM files!
This:
[root@xenlab ~]# wget --mirror http://192.168.0.1/CentOS-5.4-x86_64/ \
--directory-prefix=/tmp/centos-mini/ \
--no-host-directories --cut-dirs=1 --reject="*.rpm" \
-o /tmp/wget-log
Results in:-
Saving to: `/tmp/centos-mini/CentOS/index.html?C=M;O=A'
Saving to: `/tmp/centos-mini/CentOS/index.html?C=S;O=A'
Saving to: `/tmp/centos-mini/CentOS/index.html?C=D;O=A'
Saving to: `/tmp/centos-mini/CentOS/TRANS.TBL'
Saving to: `/tmp/centos-mini/NOTES/index.html?C=N;O=D'
Saving to: `/tmp/centos-mini/NOTES/index.html?C=M;O=A'
Saving to: `/tmp/centos-mini/NOTES/index.html?C=S;O=A'
Saving to: `/tmp/centos-mini/NOTES/index.html?C=D;O=A'
Saving to: `/tmp/centos-mini/NOTES/RELEASE-NOTES-U1-as.html'
Saving to: `/tmp/centos-mini/NOTES/RELEASE-NOTES-U1-bn.html'
Good!
I also see a lot of following. Need to investigate why wget is creating these index.html files, which does not exist on the host system / web server in the first place! (This is of very low priority though):
Saving to: `/tmp/centos-mini/192.168.0.1/CentOS-5.4-x86_64/images/xen/index.html?C=S;O=D' Saving to: `/tmp/centos-mini/192.168.0.1/CentOS-5.4-x86_64/images/xen/index.html?C=D;O=D'
Now we need to pull the required RPM files and store them manually in the CentOS directory.
for file in $(cat /tmp/minivm/installed-rpms.tmp); do \
wget http://192.168.0.1/CentOS-5.4-x86_64/CentOS/${file}.rpm \
--directory-prefix=/tmp/centos-mini/CentOS/ \
--no-host-directories --cut-dirs=1 -o /tmp/wget-log
done
Packages which can be removed from @core
Here is the list of packages which I found in CENTOS-5.4-x86_64, which I was successfully able to remove, using yum, without any damage to the OS. (about 118 packages)
aspell aspell-en atk authconfig autofs avahi avahi-compat-libdns_sd bitstream-vera-fonts bluez-gnome bluez-libs bluez-utils cairo ccid coolkey cpuspeed crash cups cups-libs desktop-file-utils dhcpv6-client dnsmasq dos2unix dosfstools ecryptfs-utils ed eject fbset finger firstboot-tui fontconfig freetype GConf2 gpm gtk2 hicolor-icon-theme htmlview ifd-egate iptables-ipv6 irda-utils irqbalance jwhois krb5-workstation ksh libdrm libICE libjpeg libnotify libpng libSM libtiff libwnck libX11 libXau libXcursor libXdmcp libXext libXfixes libXft libXi libXinerama libXrandr libXrender libXres libXt libXxf86vm mailcap man man-pages mdadm mesa-libGL microcode_ctl mkbootdisk mtools nano NetworkManager NetworkManager-glib newt notification-daemon ntsysv numactl ORBit2 pam_ccreds pam_krb5 pam_pkcs11 pam_smb pango paps pcmciautils pcsc-lite pcsc-lite-libs pinfo procmail rdate redhat-lsb redhat-menus rhpl rp-pppoe rsh sendmail setuptool slang sos specspo startup-notification syslinux system-config-network-tui system-config-securitylevel-tui tcpdump trousers unix2dos vconfig wireless-tools words wpa_supplicant xorg-x11-filesystem ypbind yp-tools yum-updatesd
References/Further Reading
Some smart people have already done the bootable CD / custom CD part. Below are some links for reference:
