CENTOS Server CD project

From WBITT's Cooker!

(Difference between revisions)
Jump to: navigation, search
(Creating the exploded tree repository, the hard way)
 
(46 intermediate revisions not shown)
Line 1: Line 1:
 +
Please send your thoughts/comments to: kamran at wbitt dot com.
 +
 +
Note: This document is out dated and is superseded by the following two:
 +
 +
* [[CentOS 5.5 x86 64 Hypervisor Edition (Minimal)]]
 +
* [[CentOS 5.5 i386 Webserver Edition (Minimal)]]
 +
 +
===Why Server CD?===
===Why Server CD?===
Lately I thought of installing a XEN CENTOS 5.5 virtual machine on one of my servers out there. I needed the exploded tree of the ISO to be available. However, when I checked centos.org website, I noticed that they stopped distributing a "Single Server CD" after CENTOS 4. I could have downloaded a CENTOS 5.5 DVD image on the server but it was kind of un-desired / not practical for two reasons.
Lately I thought of installing a XEN CENTOS 5.5 virtual machine on one of my servers out there. I needed the exploded tree of the ISO to be available. However, when I checked centos.org website, I noticed that they stopped distributing a "Single Server CD" after CENTOS 4. I could have downloaded a CENTOS 5.5 DVD image on the server but it was kind of un-desired / not practical for two reasons.
Line 11: Line 19:
* To be able to create a boot-able /installable "install CD" out of the install-tree mentioned above. So that, when I (or anyone else) need to install a minimum OS on a bare-metal machine, he doesn't have to go through downloading a full DVD.  
* To be able to create a boot-able /installable "install CD" out of the install-tree mentioned above. So that, when I (or anyone else) need to install a minimum OS on a bare-metal machine, he doesn't have to go through downloading a full DVD.  
-
===Creating the exploded tree repository, the hard way===
+
===Creating the "core" install-tree, the easy way===
-
The idea is simple. I have a CENTOS 5.4 DVD already downloaded at home (and office). To get the list of packages, I did a "core" install of CENTOS on a XEN virtual machine. (I used a kickstart file with @core mentioned in the "%packages" section. Analysing its "/root/install.log", I quickly figured out which packages got installed. (clever me!). So this is the list of packages I need from the "CentOS" directory of the install-tree. (It's about 443 packages). This resulted in an installation of about 950MB in size.  
+
The idea is simple. I have a CENTOS 5.4 DVD already downloaded at home (and office). [Note: You can also do this using just CD1 of the CENTOS/RHEL distribution]. To get the list of packages, I did a "core" install of CENTOS on a XEN virtual machine. (I used a kickstart file with @core mentioned in the "%packages" section. Analysing its "/root/install.log", I quickly figured out which packages got installed. (clever me!). So this is the list of packages I need from the "CentOS" directory of the install-tree. (It's about 443 packages). This resulted in an installation of about 950MB in size.  
Here is how I used (install.log) to generate the list of required pacakges in my repository.
Here is how I used (install.log) to generate the list of required pacakges in my repository.
Line 29: Line 37:
</pre>
</pre>
 +
<pre>
 +
cat installed-rpms.tmp | cut -d: -f2 > installed-rpms.txt
 +
</pre>
 +
 +
The contents of "installed-rpms.tmx" file are here: <b>[[Core Packages List CentOS-5.4-x86 64]]</b>
This is good enough. However, I am too much irritated by seeing bluetooth, cups, x11, and similar packages being part of the core installation. So I went one step further, and started deleting packages using yum, in a polite fashion, from this VM. This way, I was able to delete some packages, which I consider un-necessary for an installation, where only a web-server is the only desired service to run, without any backend database. (Which was the case with me.). After this surgery, I was able to bring down the disk usage to about 750 MB.  And of-course, fewer number of packages. (About 118 less rpms!). I can use that in the %post to delete the un-necessary RPMs!
This is good enough. However, I am too much irritated by seeing bluetooth, cups, x11, and similar packages being part of the core installation. So I went one step further, and started deleting packages using yum, in a polite fashion, from this VM. This way, I was able to delete some packages, which I consider un-necessary for an installation, where only a web-server is the only desired service to run, without any backend database. (Which was the case with me.). After this surgery, I was able to bring down the disk usage to about 750 MB.  And of-course, fewer number of packages. (About 118 less rpms!). I can use that in the %post to delete the un-necessary RPMs!
<pre>
<pre>
-
# yum remove aspell aspell-en atk authconfig autofs avahi avahi-compat-libdns_sd bitstream-vera-fonts bluez-gnome bluez-libs bluez-utils cairo ccid coolkey cpuspeed crash cups cups-libs desktop-file-utils dhcpv6-client dnsmasq dos2unix dosfstools ecryptfs-utilsed eject fbset finger firstboot-tui fontconfig freetype GConf2 gpm gtk2 hicolor-icon-theme htmlview ifd-egate iptables-ipv6 irda-utils irqbalance jwhois krb5-workstation ksh libdrm libICE libjpeg libnotify libpng libSM libtiff libwnck libX11 libXaulibXcursor libXdmcp libXext libXfixes libXft libXi libXinerama libXrandr libXrender libXres libXt libXxf86vm mailcap man man-pages mdadm mesa-libGL microcode_ctl mkbootdisk mtools nano NetworkManager NetworkManager-glib newt notification-daemon ntsysv numactl ORBit2 pam_ccreds pam_krb5 pam_pkcs11 pam_smb pango paps pcmciautils pcsc-lite pcsc-lite-libs pinfo procmail rdate redhat-lsb redhat-menus rhpl rp-pppoe rsh sendmail setuptool slang sos specspo startup-notification syslinux system-config-network-tui system-config-securitylevel-tui tcpdump trousers unix2dos vconfig wireless-tools words wpa_supplicant xorg-x11-filesystem ypbind yp-tools yum-updatesd
+
# yum remove aspell aspell-en atk authconfig autofs avahi avahi-compat-libdns_sd \
 +
  bitstream-vera-fonts bluez-gnome bluez-libs bluez-utils cairo ccid coolkey \
 +
  cpuspeed crash cups cups-libs desktop-file-utils dhcpv6-client dnsmasq \
 +
  dos2unix dosfstools ecryptfs-utilsed eject fbset finger firstboot-tui \
 +
  fontconfig freetype GConf2 gpm gtk2 hicolor-icon-theme htmlview ifd-egate \
 +
  iptables-ipv6 irda-utils irqbalance jwhois krb5-workstation ksh libdrm \
 +
  libICE libjpeg libnotify libpng libSM libtiff libwnck libX11 libXaulibXcursor \
 +
  libXdmcp libXext libXfixes libXft libXi libXinerama libXrandr libXrender \
 +
  libXres libXt libXxf86vm mailcap man man-pages mdadm mesa-libGL microcode_ctl \
 +
  mkbootdisk mtools nano NetworkManager NetworkManager-glib newt \
 +
  notification-daemon ntsysv numactl ORBit2 pam_ccreds pam_krb5 pam_pkcs11 \
 +
  pam_smb pango paps pcmciautils pcsc-lite pcsc-lite-libs pinfo procmail rdate \
 +
  redhat-lsb redhat-menus rhpl rp-pppoe rsh sendmail setuptool slang sos \
 +
  specspo startup-notification syslinux system-config-network-tui \
 +
  system-config-securitylevel-tui tcpdump trousers unix2dos vconfig \
 +
  wireless-tools words wpa_supplicant xorg-x11-filesystem ypbind \
 +
  yp-tools yum-updatesd
</pre>
</pre>
 +
Tip: I used the following commands to generate the long yum command above:
 +
 +
<pre>
 +
# grep Erased /var/log/yum.log | awk '{ print $5}' | sort | uniq | tr '\n' ' '
 +
</pre>
 +
 +
 +
To create my minimal repository, I first created a copy of the original install tree  skipping the CentOS directory.
 +
 +
<pre>
 +
[root@xenlab ~]# rsync -av --exclude CentOS/  \
 +
  /var/www/html/CentOS-5.4-x86_64/  /var/www/html/CentOS-5.4-x86_64-core/
 +
</pre>
 +
 +
Then I used the list of packages from the install.log file, and pulled the matching RPM files from the full install-tree into this "core" install-tree. I used the "installed-rpms.txt" file, I created a moment ago, above. The installed-rpms.txt file looks like this:
 +
 +
<pre>
 +
[root@xenlab ~]# tail /tmp/minivm/installed-rpms.txt
 +
which-2.16-7.x86_64
 +
words-3.0-9.1.noarch
 +
xorg-x11-filesystem-7.1-2.fc6.noarch
 +
yp-tools-2.9-0.1.x86_64
 +
yum-3.2.22-20.el5.centos.noarch
 +
yum-fastestmirror-1.1.16-13.el5.centos.noarch
 +
yum-metadata-parser-1.1.2-3.el5.centos.x86_64
 +
zip-2.31-2.el5.x86_64
 +
zlib-1.2.3-3.i386
 +
zlib-1.2.3-3.x86_64
 +
[root@xenlab ~]#
 +
</pre>
 +
 +
Create the CentOS directory in the "core" install tree:
 +
<pre>
 +
[root@xenlab ~]# mkdir /var/www/html/CentOS-5.4-x86_64-core/CentOS
 +
</pre>
 +
 +
Then copy the selected files from the full install-tree into this "core" tree, using the installed-rpms.txt file.
 +
 +
<pre>
 +
 +
 +
[root@xenlab ~]# for file in $(cat /tmp/minivm/installed-rpms.txt); do
 +
  cp -v /var/www/html/CentOS-5.4-x86_64/CentOS/"${file}.rpm" \
 +
    /var/www/html/CentOS-5.4-x86_64-core/CentOS/
 +
done
 +
</pre>
 +
 +
Check the "core" CentOS directory:
 +
 +
<pre>
 +
[root@xenlab ~]# cd /var/www/html/CentOS-5.4-x86_64-core/CentOS/
 +
 +
[root@xenlab CentOS]# ls -l | wc -l
 +
443
 +
 +
[root@xenlab CentOS]# du -sh .
 +
273M    .
 +
[root@xenlab CentOS]#
 +
</pre>
 +
 +
Lovely! As you can see only 443 RPMs and 273 MB occupied by the RPMs only!
 +
 +
Note: We need to (additionally) copy the normal kernel to the new location too, because someone might want to use this for installing a non-XEN machine, or for the bare-metal machine. In that case a normal kernel would be required.
 +
 +
<pre>
 +
[root@xenlab ~]# cp /var/www/html/CentOS-5.4-x86_64/CentOS/kernel-2.6.18-164.el5.x86_64.rpm \
 +
  /var/www/html/CentOS-5.4-x86_64-core/CentOS/
 +
[root@xenlab ~]# 
 +
</pre>
 +
 +
Notice the difference now:
 +
 +
<pre>
 +
[root@xenlab CentOS]# cd /var/www/html/CentOS-5.4-x86_64-core/CentOS/
 +
 +
[root@xenlab CentOS]# ls | wc -l
 +
444
 +
 +
[root@xenlab CentOS]# du -sh .
 +
291M    .
 +
</pre>
 +
 +
The complete usage of the minimal install tree is :-
 +
<pre>
 +
[root@xenlab ~]# du -sh /var/www/html/CentOS-5.4-x86_64-core
 +
606M    /var/www/html/CentOS-5.4-x86_64-core
 +
[root@xenlab ~]# 
 +
</pre>
 +
 +
 +
Note: In all the steps above, we DID NOT update/change the repository (meta) data, (using createrepo command). Nor did we change anything in the installer XML files. It works just like that.
 +
 +
====Do a test installation using this install-tree====
 +
When I was done with doing so, I created another XEN VM, and used this new (shrunk) install-tree to install the new VM to make sure it works.  I used <b>[[The minimal kickstart file]]</b> to install that system.
 +
 +
It worked! Here is the newly installed system:-
 +
 +
<pre>
 +
[root@localhost ~]# df -h
 +
Filesystem            Size  Used Avail Use% Mounted on
 +
/dev/xvda1            2.7G  986M  1.6G  39% /
 +
tmpfs                256M    0  256M  0% /dev/shm
 +
 +
[root@localhost ~]# rpm -qa | wc -l
 +
443
 +
[root@localhost ~]#
 +
</pre>
 +
 +
Then I removed the 118 packages, which I found un-necessary (as shown in the yum remove command above). The new space usage of this VM is now:
 +
 +
<pre>
 +
[root@localhost ~]# df -h
 +
Filesystem            Size  Used Avail Use% Mounted on
 +
/dev/xvda1            2.7G  775M  1.8G  31% /
 +
tmpfs                256M    0  256M  0% /dev/shm
 +
[root@localhost ~]#
 +
</pre>
-
I created a copy of the original install tree and deleted those rpms from the CentOS directory, which I did not find installed on a freshly installed machine using the "@core" group.
 
-
When I was done with doing so, I created another XEN VM, and used this new (shrunk) install-tree to install the new VM to make sure it works. And it did!
 
Now, to use this install tree on my server on the internet, I can do one of the following:
Now, to use this install tree on my server on the internet, I can do one of the following:
Line 53: Line 197:
And when I can do that, I can then create an actual bootable ISO out of it using help from many posts on the internet  (mentioned at the end of this document).
And when I can do that, I can then create an actual bootable ISO out of it using help from many posts on the internet  (mentioned at the end of this document).
-
===Packages which can be removed from @core===
+
===Creating the "core" install-tree from a http site===
-
Here is the list of packages which I found in CENTOS-5.4-x86_64, which I was successfully able to remove, using yum, without any damage to the OS. (about 118 packages)
+
Here, I am using wget to get the basic structure of CentOS full repository to my local machine, "without" pulling any rpms.
 +
 
 +
Notes:
 +
* In the example below, /tmp/centos-mini is the location where I am building the "core" install-tree.  
 +
* 192.168.0.1 is a web server, with complete DVD install-tree exported through web service.
 +
* Interestingly CentOS directory is the "only" directory in any install tree, which contains RPM files!
<pre>
<pre>
-
aspell
+
This:
-
aspell-en
+
 
-
atk
+
[root@xenlab ~]# wget --mirror  http://192.168.0.1/CentOS-5.4-x86_64/ \
-
authconfig
+
      --directory-prefix=/tmp/centos-mini/  --no-parent \
-
autofs
+
      --no-host-directories --cut-dirs=1 --reject="*.rpm" \
-
avahi
+
      -o /tmp/wget-log
-
avahi-compat-libdns_sd
+
 
-
bitstream-vera-fonts
+
Results in:-
-
bluez-gnome
+
 
-
bluez-libs
+
Saving to: `/tmp/centos-mini/CentOS/index.html?C=M;O=A'
-
bluez-utils
+
Saving to: `/tmp/centos-mini/CentOS/index.html?C=S;O=A'
-
cairo
+
Saving to: `/tmp/centos-mini/CentOS/index.html?C=D;O=A'
-
ccid
+
Saving to: `/tmp/centos-mini/CentOS/TRANS.TBL'
-
coolkey
+
Saving to: `/tmp/centos-mini/NOTES/index.html?C=N;O=D'
-
cpuspeed
+
Saving to: `/tmp/centos-mini/NOTES/index.html?C=M;O=A'
-
crash
+
Saving to: `/tmp/centos-mini/NOTES/index.html?C=S;O=A'
-
cups
+
Saving to: `/tmp/centos-mini/NOTES/index.html?C=D;O=A'
-
cups-libs
+
Saving to: `/tmp/centos-mini/NOTES/RELEASE-NOTES-U1-as.html'
-
desktop-file-utils
+
Saving to: `/tmp/centos-mini/NOTES/RELEASE-NOTES-U1-bn.html'
-
dhcpv6-client
+
-
dnsmasq
+
-
dos2unix
+
-
dosfstools
+
-
ecryptfs-utils
+
-
ed
+
-
eject
+
-
fbset
+
-
finger
+
-
firstboot-tui
+
-
fontconfig
+
-
freetype
+
-
GConf2
+
-
gpm
+
-
gtk2
+
-
hicolor-icon-theme
+
-
htmlview
+
-
ifd-egate
+
-
iptables-ipv6
+
-
irda-utils
+
-
irqbalance
+
-
jwhois
+
-
krb5-workstation
+
-
ksh
+
-
libdrm
+
-
libICE
+
-
libjpeg
+
-
libnotify
+
-
libpng
+
-
libSM
+
-
libtiff
+
-
libwnck
+
-
libX11
+
-
libXau
+
-
libXcursor
+
-
libXdmcp
+
-
libXext
+
-
libXfixes
+
-
libXft
+
-
libXi
+
-
libXinerama
+
-
libXrandr
+
-
libXrender
+
-
libXres
+
-
libXt
+
-
libXxf86vm
+
-
mailcap
+
-
man
+
-
man-pages
+
-
mdadm
+
-
mesa-libGL
+
-
microcode_ctl
+
-
mkbootdisk
+
-
mtools
+
-
nano
+
-
NetworkManager
+
-
NetworkManager-glib
+
-
newt
+
-
notification-daemon
+
-
ntsysv
+
-
numactl
+
-
ORBit2
+
-
pam_ccreds
+
-
pam_krb5
+
-
pam_pkcs11
+
-
pam_smb
+
-
pango
+
-
paps
+
-
pcmciautils
+
-
pcsc-lite
+
-
pcsc-lite-libs
+
-
pinfo
+
-
procmail
+
-
rdate
+
-
redhat-lsb
+
-
redhat-menus
+
-
rhpl
+
-
rp-pppoe
+
-
rsh
+
-
sendmail
+
-
setuptool
+
-
slang
+
-
sos
+
-
specspo
+
-
startup-notification
+
-
syslinux
+
-
system-config-network-tui
+
-
system-config-securitylevel-tui
+
-
tcpdump
+
-
trousers
+
-
unix2dos
+
-
vconfig
+
-
wireless-tools
+
-
words
+
-
wpa_supplicant
+
-
xorg-x11-filesystem
+
-
ypbind
+
-
yp-tools
+
-
yum-updatesd
+
</pre>
</pre>
 +
 +
 +
Good!
 +
 +
 +
I also see a lot of following. Need to investigate why wget is creating these index.html files, which does not exist on the host system / web server in the first place! (This is of very low priority though):
 +
 +
<pre>
 +
Saving to: `/tmp/centos-mini/192.168.0.1/CentOS-5.4-x86_64/images/xen/index.html?C=S;O=D'
 +
Saving to: `/tmp/centos-mini/192.168.0.1/CentOS-5.4-x86_64/images/xen/index.html?C=D;O=D'
 +
</pre>
 +
 +
 +
Now we need to pull the required RPM files and store them manually in the CentOS directory. 
 +
 +
<pre>
 +
for file in $(cat /tmp/minivm/installed-rpms.txt); do
 +
  wget  http://192.168.0.1/CentOS-5.4-x86_64/CentOS/${file}.rpm  \
 +
  --directory-prefix=/tmp/centos-mini/CentOS/ --no-parent  \
 +
  --no-host-directories --cut-dirs=1  -o /tmp/wget-log
 +
done 
 +
</pre>
 +
 +
Make sure you have both kernel and kernel-xen packages in this repository. Since I already have kernel-xen, I will pull kernel-2.6.18-164.el5.x86_64.rpm :
 +
 +
<pre>
 +
wget  http://192.168.0.1/CentOS-5.4-x86_64/CentOS/kernel-2.6.18-164.el5.x86_64.rpm  \
 +
  --directory-prefix=/tmp/centos-mini/CentOS/ --no-parent  \
 +
  --no-host-directories --cut-dirs=1  -o /tmp/wget-log
 +
</pre>
 +
 +
<b>Note:</b> This solution (till here), has worked on a live server, using a kickstart file with only "@core" mentioned under "%packages".  I am now working on the bootable ISO image, which is so far quite painful to do.
 +
 +
===Create a tarball of your mini install-tree for easier distribution===
 +
<pre>
 +
[root@www cdimages]# tar cjf CentOS-5.4-x86_64.tar.bz2 CentOS-5.4-x86_64
 +
 +
[root@www cdimages]# du -sh *
 +
537M    CentOS-5.4-x86_64
 +
493M    CentOS-5.4-x86_64.tar.bz2
 +
[root@www cdimages]#
 +
</pre>
 +
 +
===ToDo===
 +
I will make this available in few days on http://downloads.wbitt.com. (God Willing). Please note that this is not a bootable CD, it is just an install tree,  which would require you to use a kickstart with only three entries in the %packages section.
 +
 +
<pre>
 +
%packages
 +
@core
 +
wget
 +
yum
 +
</pre>
 +
 +
Modifying the comps.xml and updating the repository data, so far is very exciting and geeky, BUT very painful procedure.
 +
 +
This is usable and I have successfully installed few (real and virtual) machines with it.
 +
 +
P.S. The procedure to make a bootable CD out of this install tree is now complete, tested and made available here: [[Custom Linux bootable CD]]. So the bootable CD is no more under a ToDo!!!
 +
 +
===Packages which can be removed from @core===
 +
The list is moved to separate page, here: [[Packages which can be removed after Core installation]]
===References/Further Reading===
===References/Further Reading===
-
Some smart people have already done this. Below are some links for reference:
+
Some smart people have already done the bootable CD / custom CD part. Below are some links for reference:
-
* http://www.sirius.pe.kr/v5/tt
+
# http://people.redhat.com/rkeech/custom-distro.txt , which is now totally out-dated.
-
* http://smorgasbork.com/linux/35-linux/58-building-a-custom-centos-5-kickstart-disc-part-1
+
# http://greghaygood.com/2007/12/11/build-a-custom-centos-5-install-cd
-
* http://256.com/gray/docs/redhat_boot/
+
# http://cinq.com/2008/04/14/rebuilding-the-dvd-iso-for-centos-5-or-rhel-5/
-
* http://www.linuxjournal.com/article/6473
+
# http://www.linuxjournal.com/article/6473?page=1,0
-
* http://people.redhat.com/rkeech/custom-distro.txt
+

Current revision as of 07:08, 13 June 2011

Please send your thoughts/comments to: kamran at wbitt dot com.

Note: This document is out dated and is superseded by the following two:


Contents

Why Server CD?

Lately I thought of installing a XEN CENTOS 5.5 virtual machine on one of my servers out there. I needed the exploded tree of the ISO to be available. However, when I checked centos.org website, I noticed that they stopped distributing a "Single Server CD" after CENTOS 4. I could have downloaded a CENTOS 5.5 DVD image on the server but it was kind of un-desired / not practical for two reasons.

  • The centos.org distributes DVD iso image through bit-torrent, and my server service provider does not allow P2P or torrent traffic.
  • Even if I had one, I would have to download one big 4-5 GB file to my server. Whereas a few hundred MBs would do, if I want a text based "core" installation only.

So there arose an itch, and I decided to create a project to scratch that itch!

Objectives / Project goals

  • To create a repository / collection of files, a so called "Server CD" install-tree, small enough to provide the "core" installation, plus yum and wget.
  • To be able to create a boot-able /installable "install CD" out of the install-tree mentioned above. So that, when I (or anyone else) need to install a minimum OS on a bare-metal machine, he doesn't have to go through downloading a full DVD.

Creating the "core" install-tree, the easy way

The idea is simple. I have a CENTOS 5.4 DVD already downloaded at home (and office). [Note: You can also do this using just CD1 of the CENTOS/RHEL distribution]. To get the list of packages, I did a "core" install of CENTOS on a XEN virtual machine. (I used a kickstart file with @core mentioned in the "%packages" section. Analysing its "/root/install.log", I quickly figured out which packages got installed. (clever me!). So this is the list of packages I need from the "CentOS" directory of the install-tree. (It's about 443 packages). This resulted in an installation of about 950MB in size.

Here is how I used (install.log) to generate the list of required pacakges in my repository.

[root@xenlab minivm]# grep "Installing" install.log | awk '{print $2}'|sort > installed-rpms.tmp

Then I edited the resulting file to remove strange numbers before the package names, such as:

basesystem-8.0-5.1.1.el5.centos.noarch
cracklib-dicts-2.8.9-3.3.x86_64
1:termcap-5.5-1.20060701.1.noarch          <<<<------- Such as this one. What is this number ?
nash-5.1.19.6-54.x86_64
cat installed-rpms.tmp | cut -d: -f2 > installed-rpms.txt

The contents of "installed-rpms.tmx" file are here: Core Packages List CentOS-5.4-x86 64

This is good enough. However, I am too much irritated by seeing bluetooth, cups, x11, and similar packages being part of the core installation. So I went one step further, and started deleting packages using yum, in a polite fashion, from this VM. This way, I was able to delete some packages, which I consider un-necessary for an installation, where only a web-server is the only desired service to run, without any backend database. (Which was the case with me.). After this surgery, I was able to bring down the disk usage to about 750 MB. And of-course, fewer number of packages. (About 118 less rpms!). I can use that in the %post to delete the un-necessary RPMs!

# yum remove aspell aspell-en atk authconfig autofs avahi avahi-compat-libdns_sd \ 
  bitstream-vera-fonts bluez-gnome bluez-libs bluez-utils cairo ccid coolkey \ 
  cpuspeed crash cups cups-libs desktop-file-utils dhcpv6-client dnsmasq \ 
  dos2unix dosfstools ecryptfs-utilsed eject fbset finger firstboot-tui \ 
  fontconfig freetype GConf2 gpm gtk2 hicolor-icon-theme htmlview ifd-egate \ 
  iptables-ipv6 irda-utils irqbalance jwhois krb5-workstation ksh libdrm  \
  libICE libjpeg libnotify libpng libSM libtiff libwnck libX11 libXaulibXcursor \ 
  libXdmcp libXext libXfixes libXft libXi libXinerama libXrandr libXrender \ 
  libXres libXt libXxf86vm mailcap man man-pages mdadm mesa-libGL microcode_ctl \
  mkbootdisk mtools nano NetworkManager NetworkManager-glib newt \
  notification-daemon ntsysv numactl ORBit2 pam_ccreds pam_krb5 pam_pkcs11 \ 
  pam_smb pango paps pcmciautils pcsc-lite pcsc-lite-libs pinfo procmail rdate \ 
  redhat-lsb redhat-menus rhpl rp-pppoe rsh sendmail setuptool slang sos \ 
  specspo startup-notification syslinux system-config-network-tui \
  system-config-securitylevel-tui tcpdump trousers unix2dos vconfig \ 
  wireless-tools words wpa_supplicant xorg-x11-filesystem ypbind \ 
  yp-tools yum-updatesd

Tip: I used the following commands to generate the long yum command above:

# grep Erased /var/log/yum.log | awk '{ print $5}' | sort | uniq | tr '\n' ' '


To create my minimal repository, I first created a copy of the original install tree skipping the CentOS directory.

[root@xenlab ~]# rsync -av --exclude CentOS/  \ 
  /var/www/html/CentOS-5.4-x86_64/  /var/www/html/CentOS-5.4-x86_64-core/

Then I used the list of packages from the install.log file, and pulled the matching RPM files from the full install-tree into this "core" install-tree. I used the "installed-rpms.txt" file, I created a moment ago, above. The installed-rpms.txt file looks like this:

[root@xenlab ~]# tail /tmp/minivm/installed-rpms.txt
which-2.16-7.x86_64
words-3.0-9.1.noarch
xorg-x11-filesystem-7.1-2.fc6.noarch
yp-tools-2.9-0.1.x86_64
yum-3.2.22-20.el5.centos.noarch
yum-fastestmirror-1.1.16-13.el5.centos.noarch
yum-metadata-parser-1.1.2-3.el5.centos.x86_64
zip-2.31-2.el5.x86_64
zlib-1.2.3-3.i386
zlib-1.2.3-3.x86_64
[root@xenlab ~]#

Create the CentOS directory in the "core" install tree:

[root@xenlab ~]# mkdir /var/www/html/CentOS-5.4-x86_64-core/CentOS

Then copy the selected files from the full install-tree into this "core" tree, using the installed-rpms.txt file.



[root@xenlab ~]# for file in $(cat /tmp/minivm/installed-rpms.txt); do 
  cp -v /var/www/html/CentOS-5.4-x86_64/CentOS/"${file}.rpm" \ 
    /var/www/html/CentOS-5.4-x86_64-core/CentOS/ 
done 

Check the "core" CentOS directory:

[root@xenlab ~]# cd /var/www/html/CentOS-5.4-x86_64-core/CentOS/

[root@xenlab CentOS]# ls -l | wc -l
443

[root@xenlab CentOS]# du -sh .
273M    .
[root@xenlab CentOS]# 

Lovely! As you can see only 443 RPMs and 273 MB occupied by the RPMs only!

Note: We need to (additionally) copy the normal kernel to the new location too, because someone might want to use this for installing a non-XEN machine, or for the bare-metal machine. In that case a normal kernel would be required.

[root@xenlab ~]# cp /var/www/html/CentOS-5.4-x86_64/CentOS/kernel-2.6.18-164.el5.x86_64.rpm \ 
  /var/www/html/CentOS-5.4-x86_64-core/CentOS/ 
[root@xenlab ~]#  

Notice the difference now:

[root@xenlab CentOS]# cd /var/www/html/CentOS-5.4-x86_64-core/CentOS/

[root@xenlab CentOS]# ls | wc -l
444

[root@xenlab CentOS]# du -sh .
291M    .

The complete usage of the minimal install tree is :-

[root@xenlab ~]# du -sh /var/www/html/CentOS-5.4-x86_64-core
606M    /var/www/html/CentOS-5.4-x86_64-core
[root@xenlab ~]#  


Note: In all the steps above, we DID NOT update/change the repository (meta) data, (using createrepo command). Nor did we change anything in the installer XML files. It works just like that.

Do a test installation using this install-tree

When I was done with doing so, I created another XEN VM, and used this new (shrunk) install-tree to install the new VM to make sure it works. I used The minimal kickstart file to install that system.

It worked! Here is the newly installed system:-

[root@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            2.7G  986M  1.6G  39% /
tmpfs                 256M     0  256M   0% /dev/shm

[root@localhost ~]# rpm -qa | wc -l
443
[root@localhost ~]#

Then I removed the 118 packages, which I found un-necessary (as shown in the yum remove command above). The new space usage of this VM is now:

[root@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            2.7G  775M  1.8G  31% /
tmpfs                 256M     0  256M   0% /dev/shm
[root@localhost ~]#



Now, to use this install tree on my server on the internet, I can do one of the following:

  • Push the repository tarball to my XEN physical host/server on internet,
  • Or, make a script to "wget" all the required packages from a CENTOS mirror, create a repo for the installer, and make that available through yum on that server.

The first method is kind of immediately do-able, but not "exciting". It definitely involves copying all the install tree to the XEN host up there. To do the second method, I created a kind of script, which can pull the packages from any install tree on the internet and populate the CentOS directory on my own install-tree. Good enough, but still not geeky enough.

What I want to do now, is to be able to parse comps.xml file, present in the repodata directory of any regular install tree, find out the packages needed for "@core" and then pull those packages from the regular install tree into my own install tree. (This would be geeky/cool.)

And when I can do that, I can then create an actual bootable ISO out of it using help from many posts on the internet (mentioned at the end of this document).

Creating the "core" install-tree from a http site

Here, I am using wget to get the basic structure of CentOS full repository to my local machine, "without" pulling any rpms.

Notes:

  • In the example below, /tmp/centos-mini is the location where I am building the "core" install-tree.
  • 192.168.0.1 is a web server, with complete DVD install-tree exported through web service.
  • Interestingly CentOS directory is the "only" directory in any install tree, which contains RPM files!
This:

[root@xenlab ~]# wget --mirror  http://192.168.0.1/CentOS-5.4-x86_64/ \ 
      --directory-prefix=/tmp/centos-mini/  --no-parent \ 
      --no-host-directories --cut-dirs=1 --reject="*.rpm" \ 
      -o /tmp/wget-log

Results in:-

Saving to: `/tmp/centos-mini/CentOS/index.html?C=M;O=A'
Saving to: `/tmp/centos-mini/CentOS/index.html?C=S;O=A'
Saving to: `/tmp/centos-mini/CentOS/index.html?C=D;O=A'
Saving to: `/tmp/centos-mini/CentOS/TRANS.TBL'
Saving to: `/tmp/centos-mini/NOTES/index.html?C=N;O=D'
Saving to: `/tmp/centos-mini/NOTES/index.html?C=M;O=A'
Saving to: `/tmp/centos-mini/NOTES/index.html?C=S;O=A'
Saving to: `/tmp/centos-mini/NOTES/index.html?C=D;O=A'
Saving to: `/tmp/centos-mini/NOTES/RELEASE-NOTES-U1-as.html'
Saving to: `/tmp/centos-mini/NOTES/RELEASE-NOTES-U1-bn.html'


Good!


I also see a lot of following. Need to investigate why wget is creating these index.html files, which does not exist on the host system / web server in the first place! (This is of very low priority though):

Saving to: `/tmp/centos-mini/192.168.0.1/CentOS-5.4-x86_64/images/xen/index.html?C=S;O=D'
Saving to: `/tmp/centos-mini/192.168.0.1/CentOS-5.4-x86_64/images/xen/index.html?C=D;O=D'


Now we need to pull the required RPM files and store them manually in the CentOS directory.

for file in $(cat /tmp/minivm/installed-rpms.txt); do 
  wget  http://192.168.0.1/CentOS-5.4-x86_64/CentOS/${file}.rpm  \
  --directory-prefix=/tmp/centos-mini/CentOS/ --no-parent  \ 
  --no-host-directories --cut-dirs=1  -o /tmp/wget-log
done  

Make sure you have both kernel and kernel-xen packages in this repository. Since I already have kernel-xen, I will pull kernel-2.6.18-164.el5.x86_64.rpm :

wget  http://192.168.0.1/CentOS-5.4-x86_64/CentOS/kernel-2.6.18-164.el5.x86_64.rpm  \
  --directory-prefix=/tmp/centos-mini/CentOS/ --no-parent  \ 
  --no-host-directories --cut-dirs=1  -o /tmp/wget-log

Note: This solution (till here), has worked on a live server, using a kickstart file with only "@core" mentioned under "%packages". I am now working on the bootable ISO image, which is so far quite painful to do.

Create a tarball of your mini install-tree for easier distribution

[root@www cdimages]# tar cjf CentOS-5.4-x86_64.tar.bz2 CentOS-5.4-x86_64

[root@www cdimages]# du -sh *
537M    CentOS-5.4-x86_64
493M    CentOS-5.4-x86_64.tar.bz2
[root@www cdimages]# 

ToDo

I will make this available in few days on http://downloads.wbitt.com. (God Willing). Please note that this is not a bootable CD, it is just an install tree, which would require you to use a kickstart with only three entries in the %packages section.

%packages
@core
wget
yum

Modifying the comps.xml and updating the repository data, so far is very exciting and geeky, BUT very painful procedure.

This is usable and I have successfully installed few (real and virtual) machines with it.

P.S. The procedure to make a bootable CD out of this install tree is now complete, tested and made available here: Custom Linux bootable CD. So the bootable CD is no more under a ToDo!!!

Packages which can be removed from @core

The list is moved to separate page, here: Packages which can be removed after Core installation

References/Further Reading

Some smart people have already done the bootable CD / custom CD part. Below are some links for reference:

  1. http://people.redhat.com/rkeech/custom-distro.txt , which is now totally out-dated.
  2. http://greghaygood.com/2007/12/11/build-a-custom-centos-5-install-cd
  3. http://cinq.com/2008/04/14/rebuilding-the-dvd-iso-for-centos-5-or-rhel-5/
  4. http://www.linuxjournal.com/article/6473?page=1,0
Personal tools