Custom Linux bootable CD

From WBITT's Cooker!

(Difference between revisions)
Jump to: navigation, search
(Required Packages on the build host)
 
(31 intermediate revisions not shown)
Line 1: Line 1:
-
This article is in relation to the article [[CENTOS Server CD project]] , and is "heavily" based on http://people.redhat.com/rkeech/custom-distro.txt . So here it goes.  
+
Please send your thoughts/comments to: kamran at wbitt dot com.
 +
 
 +
This article is in relation to the article [[CENTOS Server CD project]]. I took some concepts from :
 +
 
 +
# http://people.redhat.com/rkeech/custom-distro.txt , which is now totally out-dated.  
 +
# http://greghaygood.com/2007/12/11/build-a-custom-centos-5-install-cd
 +
 
 +
Note: In recent versions of RedHat and CentOS, most of the steps/commands shown in link #1 above, are now taken care by a single "createrepo" command.
===First, the setup===
===First, the setup===
-
My machine, on which I am doing all these steps / building the Custom CENTOS CD, is named kworkhorse, but will be referred to as the role <b><i>buildhost</i></b> in this text. It runs Fedora 11 64 bit.  
+
Originally I tried to use my Fedora-11 machine (named kworkhorse), for doing all these steps of  building the Custom CentOS cd. However, after burning two weeks of time, I realized that I have to build CentOS CD from a CentOS build host.
-
I have a directory <b>/data/cdimages</b>, which contains full install tree of original CENTOS 5.4 64 bit CD in <b>CENTOS-5.4-x86_64</b> subdirectory, and an empty <b>CENTOS-5.4-x86_64-core</b> (build) directory. The directory layout looks like the following:
+
So what I did is, I simply created a small virtual machine, within my Fedora 11, which runs CentOS-5.4-x86_64. It is named <b>"buildhost"</b>. I assigned 1GB of RAM and 3 GB of diskspace to this virtual machine. I did not use more space, because, I wanted to use least amount of disk space, and kind of re-use the already prepared "build-tree" from the Fedora11 machine. What I did was export /data/cdimages from Fedora11 host using NFS and mounted that on /mnt/cdimages on the newly  CentOS VM.
-
<pre>
+
The directory <b>/data/cdimages</b>, on my Fedora machine, contains full install tree of original CENTOS 5.4 64 bit CD in <b>CENTOS-5.4-x86_64</b> subdirectory, and an empty <b>CENTOS-5.4-x86_64-core</b> (build) directory. When this directory is mounted on my buildhost, over NFS, the directory layout looks like the following:
-
[root@kworkhorse cdimages]# pwd
+
<pre>
-
/data/cdimages
+
[root@buildhost cdimages]# pwd
 +
/mnt/cdimages
-
[root@kworkhorse cdimages]# tree -d
+
[root@buildhost cdimages]# tree -d
.
.
├── CentOS-5.4-x86_64
├── CentOS-5.4-x86_64
Line 24: Line 32:
└── CentOS-5.4-x86_64-core
└── CentOS-5.4-x86_64-core
-
[root@kworkhorse cdimages]#
 
</pre>
</pre>
-
The directory <b>/data/cdimages/CentOS-5.4-x86_64-core</b> (which is empty at the moment), will be the custom install tree, and referred to as <b>$RHROOT</b> in this document. Once again:
+
The directory <b>/mnt/cdimages/CentOS-5.4-x86_64-core</b> (which is empty at the moment), will be the custom install tree, and will be referred to as <b>build-tree</b> or <b>build-directory</b> in this document. Once again:
<b>
<b>
-
* BuildHost: kworkhorse
+
* Machine on which CentOS CD is built: BuildHost
-
* BuildDirectory ( $RHROOT): /data/cdimages/CentOS-5.4-x86_64-core
+
* OS on build host: CentOS-5.4-x86_64
 +
* BuildDirectory: /mnt/cdimages/CentOS-5.4-x86_64-core
</b>
</b>
Line 41: Line 49:
</pre>
</pre>
-
Note: anaconda-runtime is RHEL/CENTOS specific package, and is part of the RHEL/CENTOS distribution. As evident below:
+
Note: anaconda-runtime is RHEL/CENTOS specific package, and is part of the RHEL/CENTOS distribution.  
-
<pre>
 
-
[kamran@kworkhorse CentOS]$ ls /data/cdimages/CentOS-5.4-x86_64/CentOS/anaconda-* -1
 
-
anaconda-11.1.2.195-1.el5.centos.x86_64.rpm
 
-
anaconda-runtime-11.1.2.195-1.el5.centos.x86_64.rpm
 
-
[kamran@kworkhorse CentOS]$
 
-
</pre>
 
-
This package "anaconda-runtime" provides the following packages:
+
Off-the-topic: On Fedora, if you want to generate custom fedora spin-offs, you can use "pungi" and "revisor" . It has a wizard-like GUI interface.
<pre>
<pre>
-
[kamran@kworkhorse CentOS]$ rpm -qpl anaconda-runtime-11.1.2.195-1.el5.centos.x86_64.rpm
+
yum -y install pungi revisor
-
. . .
+
-
/usr/lib/anaconda-runtime
+
-
/usr/lib/anaconda-runtime/boot
+
-
/usr/lib/anaconda-runtime/boot/boot.msg
+
-
/usr/lib/anaconda-runtime/boot/general.msg
+
-
/usr/lib/anaconda-runtime/boot/options.msg
+
-
/usr/lib/anaconda-runtime/boot/param.msg
+
-
/usr/lib/anaconda-runtime/boot/rescue.msg
+
-
/usr/lib/anaconda-runtime/boot/syslinux.cfg
+
-
/usr/lib/anaconda-runtime/buildinstall
+
-
. . .
+
-
. . .
+
-
/usr/lib/anaconda-runtime/pyrc.pyo
+
-
/usr/lib/anaconda-runtime/readmap
+
-
/usr/lib/anaconda-runtime/screenfont-x86_64.gz
+
-
/usr/lib/anaconda-runtime/scrubtree
+
-
/usr/lib/anaconda-runtime/splittree.py
+
-
/usr/lib/anaconda-runtime/splittree.pyc
+
-
/usr/lib/anaconda-runtime/splittree.pyo
+
-
/usr/lib/anaconda-runtime/trimmodalias
+
-
/usr/lib/anaconda-runtime/trimpciids
+
-
/usr/lib/anaconda-runtime/upd-instroot
+
-
/usr/lib/anaconda-runtime/yumcache
+
-
. . .
+
-
[kamran@kworkhorse CentOS]$
+
</pre>
</pre>
-
 
-
 
-
If you are using Fedora, you may not find anaconda-runtime. Instead anaconda and anaconda-runtime are (kind of) merged into single package, which provides (almost) the same functionality :
 
-
 
-
<pre>
 
-
[root@kworkhorse cdimages]# rpm -ql anaconda  | grep runtime
 
-
. . .
 
-
/usr/lib/anaconda-runtime
 
-
/usr/lib/anaconda-runtime/boot
 
-
/usr/lib/anaconda-runtime/boot/boot.msg
 
-
/usr/lib/anaconda-runtime/boot/grub.conf
 
-
/usr/lib/anaconda-runtime/boot/syslinux.cfg
 
-
/usr/lib/anaconda-runtime/buildinstall
 
-
/usr/lib/anaconda-runtime/buildinstall.functions
 
-
/usr/lib/anaconda-runtime/genmodinfo
 
-
/usr/lib/anaconda-runtime/getkeymaps
 
-
/usr/lib/anaconda-runtime/keymaps-override-x86_64
 
-
/usr/lib/anaconda-runtime/loader
 
-
/usr/lib/anaconda-runtime/loader/init
 
-
/usr/lib/anaconda-runtime/loader/loader
 
-
. . .
 
-
. . .
 
-
/usr/lib/anaconda-runtime/pyrc.pyc
 
-
/usr/lib/anaconda-runtime/pyrc.pyo
 
-
/usr/lib/anaconda-runtime/readmap
 
-
/usr/lib/anaconda-runtime/screenfont-x86_64.gz
 
-
/usr/lib/anaconda-runtime/scrubtree
 
-
/usr/lib/anaconda-runtime/trimpciids
 
-
/usr/lib/anaconda-runtime/upd-instroot
 
-
/usr/lib/anaconda-runtime/upd-updates
 
-
. . .
 
-
[root@kworkhorse cdimages]#
 
-
</pre>
 
-
 
-
You may need the following additional packages on more recent distributions. On my build host (Fedora 11 64 bit), I had to install "pungi" to get the "pkgorder" command/tool.
 
-
 
-
<pre>
 
-
yum -y install pungi
 
-
</pre>
 
-
 
-
Optional: On Fedora, if you want to generate custom fedora spin-offs, you can use "revisor" . It has a wizard-like GUI interface.
 
-
 
-
<pre>
 
-
yum -y install revisor
 
-
</pre>
 
-
 
-
===The environment variables===
 
-
 
-
Quote from the original Howto:
 
-
<i>Many of the scripts provided in these packages are Python scripts.  For them to work properly the PYTHONPATH variable must be set to /usr/lib/anaconda and be exported in the shell environment in which the commands are run.  Many of the scripts are from the directory /usr/lib/anaconda-runtime, so either put this in the PATH or provide an absolute path to commands.</i>
 
-
 
-
So, you should setup the necessary environment variables. I would advise you to add the following two lines to your ~/.bash_profile, so each time you open a terminal, these values are set.
 
-
 
-
<pre>
 
-
export RHROOT=/data/cdimages/CentOS-5.4-x86_64-core
 
-
export PYTHONPATH=/usr/lib/anaconda
 
-
export PATH=$PATH:/usr/lib/anaconda-runtime
 
-
</pre>
 
-
 
-
<b>Note:</b> It is important to note, that even on a 64 bit system (as I have), these two paths are /usr/lib/anaconda and /usr/lib/anacoda-runtime. i.e. These two directories does not exist in /usr/lib64.
 
===Build the install tree===
===Build the install tree===
Line 146: Line 63:
<pre>
<pre>
# rsync -av --exclude CentOS/  \  
# rsync -av --exclude CentOS/  \  
-
   /data/cdimages/CentOS-5.4-x86_64/  /data/cdimages/CentOS-5.4-x86_64-core/
+
   /mnt/cdimages/CentOS-5.4-x86_64/  /mnt/cdimages/CentOS-5.4-x86_64-core/
</pre>
</pre>
 +
 +
Note: I prefer rsync, because it copies the .dot-files too. Later, we would be needing a special file named .discinfo, from the original tree.
When done, your buildtree would look like this:
When done, your buildtree would look like this:
<pre>
<pre>
-
[root@kworkhorse CentOS-5.4-x86_64-core]# pwd  
+
[root@buildhost CentOS-5.4-x86_64-core]# pwd  
-
/data/cdimages/CentOS-5.4-x86_64-core
+
/mnt/cdimages/CentOS-5.4-x86_64-core
-
[root@kworkhorse CentOS-5.4-x86_64-core]# tree -d
+
[root@buildhost CentOS-5.4-x86_64-core]# tree -d
.
.
├── images
├── images
Line 165: Line 84:
6 directories
6 directories
-
[root@kworkhorse CentOS-5.4-x86_64-core]#  
+
[root@buildhost CentOS-5.4-x86_64-core]#  
</pre>
</pre>
-
Now we need to copy selected RPMs from the source directory into the build directory. I created this file, by first creating a test VM using minimal Kickstart, and noted the list of packages installed in it. I could not think of any easier way. The file is located in /tmp/minivm, as installed-rpms.txt . (Look in [[CENTOS Server CD project]] to see how I generated it. Or use the [[Core Packages List CentOS-5.4-x86 64]] list for list of packages required for CentOS 5.4 64 bit.)
+
Now we need to copy selected RPMs from the source directory into the build directory. I created this file, by first creating a test VM using minimal kickstart, and noted the list of packages installed in it. I could not think of any easier way. The file is located in /tmp/minivm, as "installed-rpms.txt" . (Look in [[CENTOS Server CD project]] to see how I generated it. Or use the [[Core Packages List CentOS-5.4-x86 64]] list for list of packages required for CentOS 5.4 64 bit.)
<pre>
<pre>
-
# mkdir /data/cdimages/CentOS-5.4-x86_64-core/CentOS/
+
# mkdir /mnt/cdimages/CentOS-5.4-x86_64-core/CentOS/
# for filename in $(cat /tmp/minivm/installed-rpms.txt); do  
# for filename in $(cat /tmp/minivm/installed-rpms.txt); do  
-
   cp -v /data/cdimages/CentOS-5.4-x86_64/CentOS/"${filename}.rpm" \  
+
   cp -v /mnt/cdimages/CentOS-5.4-x86_64/CentOS/"${filename}.rpm" \  
-
     /data/cdimages/CentOS-5.4-x86_64-core/CentOS/  
+
     /mnt/cdimages/CentOS-5.4-x86_64-core/CentOS/  
done
done
</pre>
</pre>
 +
 +
Make sure you copy kernel-* files to the build tree. This is important, because at times, you may be using this CD in all sorts install scenarios, such as installation of bare-metal hosts, KVM hosts, XEN hosts, etc. So you would need at least two types of kernels in your installation.
===comps.xml===
===comps.xml===
-
From the original howot: <i>Comps file.  The file $RHROOT/RedHat/base/comps.xml defines the set of packages known to the installer.  The set of packages described in comps.xml must be reduced to  only those required.  It is not necessary to remove the unused package files from $RHROOT/RedHat/RPMS/. Some packages from $RHROOT/RedHat/RPMS/ are necessary in the process of building the install CD even though they might be excluded from the custom build.</i>
+
From the original howto: <i>Comps file.  The file $RHROOT/RedHat/base/comps.xml defines the set of packages known to the installer.  The set of packages described in comps.xml must be reduced to  only those required.  It is not necessary to remove the unused package files from $RHROOT/RedHat/RPMS/. Some packages from $RHROOT/RedHat/RPMS/ are necessary in the process of building the install CD even though they might be excluded from the custom build.</i>
Here is an <b>[[example comps.xml file]]</b> from the original howto (RHEL3 based).
Here is an <b>[[example comps.xml file]]</b> from the original howto (RHEL3 based).
-
As I mentioned in the related article, (borrowing the term from another author), th comps.xml has gazillion languages represented in it. I want to at least get rid of the languages stuff from the file, so I can work on it.
+
As I mentioned in the related article, (borrowing the sentence/term from another author), "the comps.xml has gazillion languages represented in it". I want to at least get rid of the languages stuff from the file, so I can work on it.
Here is something interesting. The original comps.xml file (CentOS-5.4) is 11418 lines.  
Here is something interesting. The original comps.xml file (CentOS-5.4) is 11418 lines.  
<pre>
<pre>
-
[root@kworkhorse CentOS-5.4-x86_64-core]# wc -l repodata/comps.xml  
+
[root@buildhost CentOS-5.4-x86_64-core]# wc -l repodata/comps.xml  
11418 repodata/comps.xml
11418 repodata/comps.xml
-
[root@kworkhorse CentOS-5.4-x86_64-core]#
+
[root@buildhost CentOS-5.4-x86_64-core]#
</pre>
</pre>
Line 198: Line 119:
<pre>
<pre>
-
[root@kworkhorse CentOS-5.4-x86_64-core]# grep  "xml:lang" repodata/comps.xml | wc -l
+
[root@buildhost CentOS-5.4-x86_64-core]# grep  "xml:lang" repodata/comps.xml | wc -l
8811
8811
</pre>
</pre>
Line 207: Line 128:
     <name xml:lang="nl">Cluster-opslag</name>
     <name xml:lang="nl">Cluster-opslag</name>
. . .  
. . .  
-
. . .
 
     <description xml:lang="nl">Pakketten die ondersteuning bieden voor cluster-opslag.</description>
     <description xml:lang="nl">Pakketten die ondersteuning bieden voor cluster-opslag.</description>
</pre>
</pre>
Line 214: Line 134:
<pre>
<pre>
-
[root@kworkhorse CentOS-5.4-x86_64-core]# grep -v  "xml:lang" repodata/comps.xml > /tmp/no-lang.comps.xml
+
[root@buildhost CentOS-5.4-x86_64-core]# grep -v  "xml:lang" repodata/comps.xml > /tmp/no-lang.comps.xml
-
[root@kworkhorse CentOS-5.4-x86_64-core]# wc -l /tmp/no-lang.comps.xml  
+
[root@buildhost CentOS-5.4-x86_64-core]# wc -l /tmp/no-lang.comps.xml  
2607 /tmp/no-lang.comps.xml
2607 /tmp/no-lang.comps.xml
Line 228: Line 148:
<pre>
<pre>
-
[root@kworkhorse CentOS-5.4-x86_64-core]# pwd
+
[root@buildhost CentOS-5.4-x86_64-core]# pwd
-
/data/cdimages/CentOS-5.4-x86_64-core
+
/mnt/cdimages/CentOS-5.4-x86_64-core
-
[root@kworkhorse CentOS-5.4-x86_64-core]# rm repodata/* -f
+
[root@buildhost CentOS-5.4-x86_64-core]# rm repodata/* -f
-
[root@kworkhorse CentOS-5.4-x86_64-core]# cp /tmp/no-lang.comps.xml  repodata/comps.xml
+
[root@buildhost CentOS-5.4-x86_64-core]# cp /tmp/no-lang.comps.xml  repodata/comps.xml
</pre>
</pre>
 +
 +
===Building the stuff===
 +
The RHEL3 based howto recommends to run the commands pkgorder, buildinstall, etc, on the build tree. This is no longer valid. A simple createrepo command takes care of everything.
 +
 +
====Testing the RPMs for any dependency errors====
 +
It is a good idea to do a test install of your RPMs, in a test directory. This will point out any dependency problems.
 +
 +
<pre>
 +
 +
[root@buildhost CentOS]# pwd
 +
/mnt/cdimages/CentOS-5.4-x86_64-core/CentOS
 +
 +
[root@buildhost CentOS]# mkdir /tmp/testrpmdb
 +
 +
[root@buildhost CentOS]# rpm --initdb --dbpath /tmp/testrpmdb/
 +
 +
[root@buildhost CentOS]# rpm --test --dbpath /tmp/testrpmdb/ -ivh *.rpm
 +
warning: acl-2.2.39-3.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID e8562897
 +
Preparing...                ########################################### [100%]
 +
[root@buildhost CentOS]#
 +
 +
[root@buildhost CentOS]# echo $?
 +
0
 +
 +
[root@buildhost CentOS]# du -sh /tmp/testrpmdb/
 +
640K    /tmp/testrpmdb/
 +
</pre>
 +
 +
Had there be a package in that directory, which would be dependent on some other packages, then you would definitely get errors, which you can fix. For an example, I copied the rpm for httpd in this build directory.
 +
 +
<pre>
 +
[root@buildhost CentOS]# cp /mnt/cdimages/CentOS-5.4-x86_64/CentOS/httpd-2.2.3-31.el5.centos.x86_64.rpm .
 +
</pre>
 +
 +
And then, I run the RPM test install again:
 +
 +
<pre>
 +
[root@buildhost CentOS]# rpm --test --dbpath /tmp/testrpmdb/ -ivh *.rpm
 +
warning: acl-2.2.39-3.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID e8562897
 +
error: Failed dependencies:
 +
        libapr-1.so.0()(64bit) is needed by httpd-2.2.3-31.el5.centos.x86_64
 +
        libaprutil-1.so.0()(64bit) is needed by httpd-2.2.3-31.el5.centos.x86_64
 +
[root@buildhost CentOS]#
 +
</pre>
 +
 +
As you can see, in the above example, either you remove the httpd RPM, or add libapr libraries too, to the CentOS directory inside your build directory. I am sure, you are intelligent enough to figure out the rest.
 +
 +
<b>Note:</b> Some documents on the net recommend that you copythe following in the CentOS directory in your build-tree, as it is suspected that they are needed for the anaconda installer to run properly. However, please note that I was able to create a CD <b>"without"</b> these RPMs , and was also able to install a VM using the the GUI of anaconda installer. These (not needed) packages are :
 +
 +
<pre>
 +
anaconda
 +
anaconda-runtime
 +
anaconda-help
 +
anaconda-product
 +
busybox
 +
busybox-anaconda
 +
memtest86+
 +
fonts-xorg-base
 +
xorg-x11
 +
joe
 +
</pre>
 +
 +
====createrepo====
 +
Now is the time to run  createrepo. Before you run that, you need to get some information from the .discinfo file. It is included in the root directory of the CentOS/RHEL  CD/DVD. If you did an rsync (or wget) to get these files from the source location, you should have it in the top level of your build-directory. Here is how it looks like:
 +
 +
<pre>
 +
[root@buildhost CentOS-5.4-x86_64-core]# pwd
 +
/mnt/cdimages/CentOS-5.4-x86_64-core
 +
 +
[root@buildhost CentOS-5.4-x86_64-core]# cat .discinfo
 +
1254416275.453877
 +
Final
 +
x86_64
 +
1,2,3,4,5,6,7
 +
CentOS/base
 +
/home/buildcentos/CENTOS/5.4/en/x86_64/CentOS
 +
CentOS/pixmaps
 +
</pre>
 +
 +
Lets move forward.
 +
 +
<pre>
 +
[root@buildhost CentOS-5.4-x86_64-core]# export discinfo=$(head -1 /mnt/cdimages/CentOS-5.4-x86_64-core/.discinfo)
 +
 +
[root@buildhost CentOS-5.4-x86_64-core]# echo $discinfo
 +
1254416275.453877
 +
</pre>
 +
 +
Now, run the createrepo command. Remember that we removed everything in the repodata directory and only placed our (very small/trimmed down), comps.xml in that directory.
 +
 +
<pre>
 +
[root@buildhost CentOS-5.4-x86_64-core]# createrepo -u "media://$discinfo" \
 +
  -g repodata/comps.xml /mnt/cdimages/CentOS-5.4-x86_64-core
 +
 +
444/444 - CentOS/mailx-8.1.1-44.2.2.x86_64.rpm                                 
 +
Saving Primary metadata
 +
Saving file lists metadata
 +
Saving other metadata
 +
[root@buildhost CentOS-5.4-x86_64-core]#
 +
</pre>
 +
 +
====Create Bootable ISO file====
 +
If the previous step did not give you any errors, you are now ready to create the actual bootable CD / ISO file out of this build-tree. (There is nothing worse then dead air!)
 +
 +
<pre>
 +
[root@buildhost CentOS-5.4-x86_64-core]# export isofilename=/mnt/cdimages/CentOS-5.4-x86_64-core.iso
 +
 +
[root@buildhost CentOS-5.4-x86_64-core]# mkisofs -r -R -J -T -v  \
 +
  -no-emul-boot -boot-load-size 4 -boot-info-table \
 +
  -V "CentOS-5u4-core 64 bit" -p "Kamran"  -A "CentOS-5u4-core 64 bit-2010/07/02" \
 +
  -b isolinux/isolinux.bin -c isolinux/boot.cat  -x "lost+found" \
 +
  -o $isofilename /mnt/cdimages/CentOS-5.4-x86_64-core
 +
 +
INFO:  UTF-8 character encoding detected by locale settings.
 +
        Assuming UTF-8 encoded filenames on source filesystem,
 +
        use -input-charset to override.
 +
mkisofs 2.01 (cpu-pc-linux-gnu)
 +
Scanning /mnt/cdimages/CentOS-5.4-x86_64-core
 +
Scanning /mnt/cdimages/CentOS-5.4-x86_64-core/images
 +
. . .
 +
 +
95.65% done, estimate finish Sat Jul  3 20:20:09 2010
 +
97.60% done, estimate finish Sat Jul  3 20:20:10 2010
 +
99.56% done, estimate finish Sat Jul  3 20:20:08 2010
 +
Total translation table size: 146726
 +
Total rockridge attributes bytes: 62153
 +
Total directory bytes: 98304
 +
Path table size(bytes): 112
 +
Done with: The File(s)                            Block(s)    255876
 +
Writing:  Ending Padblock                        Start Block 255991
 +
Done with: Ending Padblock                        Block(s)    150
 +
Max brk space used c2000
 +
256141 extents written (500 MB)
 +
[root@buildhost CentOS-5.4-x86_64-core]#
 +
</pre>
 +
 +
====Implant MD5SUM in the ISO file====
 +
The following will implant the ISO file's MD5 inside the ISO file itself.
 +
 +
<pre>
 +
[root@buildhost CentOS-5.4-x86_64-core]# implantisomd5 $isofilename
 +
Inserting md5sum into iso image...
 +
md5 = 48a3456517149f584ae81f793b933459
 +
Inserting fragment md5sums into iso image...
 +
fragmd5 = 6cae56578bb59bb8bd2289af35fa7c4f4468658de47f45e6d488be9b6b33
 +
frags = 20
 +
Setting supported flag to 0
 +
[root@buildhost CentOS-5.4-x86_64-core]#
 +
</pre>
 +
 +
===Test the stuff you just build (the bootable ISO image)===
 +
Ok, now the CD is created, as a result of last (mkisofs) step. I tested it to create a VM , on my Fedora 11 KVM host. It worked without a problem, in both text and GUI mode. Here are a few screen shots!
 +
 +
 +
[[File:Custom-cd-text-installer-small.png]]
 +
 +
[[File:Custom-cd-gui-installer-1-small.png]]    [[File:Custom-cd-gui-installer-2-small.png]]    [[File:Custom-cd-gui-installer-3-small.png]]
 +
 +
===DONE===
 +
Here is some of the information from the newly installed VM. You can see that only ssh service is running, and the install size of the system is 701MB in total.
 +
 +
<pre>
 +
[kamran@kworkhorse cdimages]$ ssh root@192.168.122.61
 +
The authenticity of host '192.168.122.61 (192.168.122.61)' can't be established.
 +
RSA key fingerprint is a1:0e:0b:01:92:6a:69:14:e6:62:6d:83:6d:08:16:1d.
 +
Are you sure you want to continue connecting (yes/no)? yes
 +
Warning: Permanently added '192.168.122.61' (RSA) to the list of known hosts.
 +
root@192.168.122.61's password:
 +
Last login: Sat Jul  3 20:53:08 2010
 +
 +
 +
[root@custom-cd ~]# netstat -antp
 +
Active Internet connections (servers and established)
 +
Proto Recv-Q Send-Q Local Address              Foreign Address            State      PID/Program name 
 +
tcp        0      0 :::22                      :::*                        LISTEN      1736/sshd         
 +
tcp        0      0 ::ffff:192.168.122.61:22    ::ffff:192.168.122.1:48069  ESTABLISHED 2154/0             
 +
 +
 +
[root@custom-cd ~]# df -h
 +
Filesystem            Size  Used Avail Use% Mounted on
 +
/dev/hda1            2.7G  701M  1.9G  28% /
 +
tmpfs                250M    0  250M  0% /dev/shm
 +
[root@custom-cd ~]#
 +
</pre>
 +
 +
The first thing, which you might want to do on this newly installed system is update it full, to bring it upto the latest version of kernel and the distribution. The benefit of such small installation is that the update will pull relatively less amount of packages, compared to a larger install. If you are connected to the internet, this should complete in minutes. Later, whatever packages you want to add, they will be pulled from the updates repository (if you have it configured that way).
 +
 +
<pre>
 +
[root@custom-cd ~]#  yum -y update
 +
</pre>
 +
 +
 +
 +
Note: If you want, you can further remove extra packages from the newly installed system. You can use the following as a guide line: [[Packages which can be removed after Core installation]] .
 +
 +
===Further reading===
 +
# http://cinq.com/2008/04/14/rebuilding-the-dvd-iso-for-centos-5-or-rhel-5/
 +
# http://www.linuxjournal.com/article/6473?page=1,0
 +
# http://www-oss.fnal.gov/projects/fermilinux/common/anaconda.comps.html
 +
# http://fedoraproject.org/wiki/PackageMaintainers/CompsXml
 +
# http://createrepo.baseurl.org/
 +
# http://mirror.centos.org/centos/4/build/distro/build.sh.txt

Current revision as of 22:23, 15 July 2010

Please send your thoughts/comments to: kamran at wbitt dot com.

This article is in relation to the article CENTOS Server CD project. I took some concepts from :

  1. http://people.redhat.com/rkeech/custom-distro.txt , which is now totally out-dated.
  2. http://greghaygood.com/2007/12/11/build-a-custom-centos-5-install-cd

Note: In recent versions of RedHat and CentOS, most of the steps/commands shown in link #1 above, are now taken care by a single "createrepo" command.

Contents

First, the setup

Originally I tried to use my Fedora-11 machine (named kworkhorse), for doing all these steps of building the Custom CentOS cd. However, after burning two weeks of time, I realized that I have to build CentOS CD from a CentOS build host.

So what I did is, I simply created a small virtual machine, within my Fedora 11, which runs CentOS-5.4-x86_64. It is named "buildhost". I assigned 1GB of RAM and 3 GB of diskspace to this virtual machine. I did not use more space, because, I wanted to use least amount of disk space, and kind of re-use the already prepared "build-tree" from the Fedora11 machine. What I did was export /data/cdimages from Fedora11 host using NFS and mounted that on /mnt/cdimages on the newly CentOS VM.

The directory /data/cdimages, on my Fedora machine, contains full install tree of original CENTOS 5.4 64 bit CD in CENTOS-5.4-x86_64 subdirectory, and an empty CENTOS-5.4-x86_64-core (build) directory. When this directory is mounted on my buildhost, over NFS, the directory layout looks like the following:

[root@buildhost cdimages]# pwd
/mnt/cdimages

[root@buildhost cdimages]# tree -d
.
├── CentOS-5.4-x86_64
│   ├── CentOS
│   ├── images
│   │   ├── pxeboot
│   │   └── xen
│   ├── isolinux
│   ├── NOTES
│   └── repodata
└── CentOS-5.4-x86_64-core

The directory /mnt/cdimages/CentOS-5.4-x86_64-core (which is empty at the moment), will be the custom install tree, and will be referred to as build-tree or build-directory in this document. Once again:

  • Machine on which CentOS CD is built: BuildHost
  • OS on build host: CentOS-5.4-x86_64
  • BuildDirectory: /mnt/cdimages/CentOS-5.4-x86_64-core

Required Packages on the build host

Before you continue, you must install the following packages on your base OS.

yum -y install anaconda anaconda-runtime mkisofs cdrecord

Note: anaconda-runtime is RHEL/CENTOS specific package, and is part of the RHEL/CENTOS distribution.


Off-the-topic: On Fedora, if you want to generate custom fedora spin-offs, you can use "pungi" and "revisor" . It has a wizard-like GUI interface.

yum -y install pungi revisor

Build the install tree

To create my minimal install tree, I first created a copy of the original install tree skipping the CentOS directory.

# rsync -av --exclude CentOS/  \ 
  /mnt/cdimages/CentOS-5.4-x86_64/  /mnt/cdimages/CentOS-5.4-x86_64-core/

Note: I prefer rsync, because it copies the .dot-files too. Later, we would be needing a special file named .discinfo, from the original tree.

When done, your buildtree would look like this:

[root@buildhost CentOS-5.4-x86_64-core]# pwd 
/mnt/cdimages/CentOS-5.4-x86_64-core

[root@buildhost CentOS-5.4-x86_64-core]# tree -d
.
├── images
│   ├── pxeboot
│   └── xen
├── isolinux
├── NOTES
└── repodata

6 directories
[root@buildhost CentOS-5.4-x86_64-core]# 

Now we need to copy selected RPMs from the source directory into the build directory. I created this file, by first creating a test VM using minimal kickstart, and noted the list of packages installed in it. I could not think of any easier way. The file is located in /tmp/minivm, as "installed-rpms.txt" . (Look in CENTOS Server CD project to see how I generated it. Or use the Core Packages List CentOS-5.4-x86 64 list for list of packages required for CentOS 5.4 64 bit.)

# mkdir /mnt/cdimages/CentOS-5.4-x86_64-core/CentOS/

# for filename in $(cat /tmp/minivm/installed-rpms.txt); do 
  cp -v /mnt/cdimages/CentOS-5.4-x86_64/CentOS/"${filename}.rpm" \ 
    /mnt/cdimages/CentOS-5.4-x86_64-core/CentOS/ 
done

Make sure you copy kernel-* files to the build tree. This is important, because at times, you may be using this CD in all sorts install scenarios, such as installation of bare-metal hosts, KVM hosts, XEN hosts, etc. So you would need at least two types of kernels in your installation.

comps.xml

From the original howto: Comps file. The file $RHROOT/RedHat/base/comps.xml defines the set of packages known to the installer. The set of packages described in comps.xml must be reduced to only those required. It is not necessary to remove the unused package files from $RHROOT/RedHat/RPMS/. Some packages from $RHROOT/RedHat/RPMS/ are necessary in the process of building the install CD even though they might be excluded from the custom build.

Here is an example comps.xml file from the original howto (RHEL3 based).

As I mentioned in the related article, (borrowing the sentence/term from another author), "the comps.xml has gazillion languages represented in it". I want to at least get rid of the languages stuff from the file, so I can work on it.

Here is something interesting. The original comps.xml file (CentOS-5.4) is 11418 lines.

[root@buildhost CentOS-5.4-x86_64-core]# wc -l repodata/comps.xml 
11418 repodata/comps.xml
[root@buildhost CentOS-5.4-x86_64-core]#

Interestingly, there are 8811 "xml:lang" lines in it!

[root@buildhost CentOS-5.4-x86_64-core]# grep  "xml:lang" repodata/comps.xml | wc -l
8811

The "xml:lang" lines look like this and repeat themselves a lot!

    <name xml:lang="nl">Cluster-opslag</name>
. . . 
    <description xml:lang="nl">Pakketten die ondersteuning bieden voor cluster-opslag.</description>

Lets create a "language-free" version of this file.

[root@buildhost CentOS-5.4-x86_64-core]# grep -v  "xml:lang" repodata/comps.xml > /tmp/no-lang.comps.xml

[root@buildhost CentOS-5.4-x86_64-core]# wc -l /tmp/no-lang.comps.xml 
2607 /tmp/no-lang.comps.xml

As you can see, only 2607 lines to edit! Great! It's time to edit this file, remove anything which is not related to us, and replace the repodata/comps.xml file with the new one.

Here is the (80 lines) file I ended with: Comps.xml for CentOS-5.4-x86 64 minimal install !!!

Now remove all files in repodata/* and copy your newly created file to repodata, as comps.xml .

[root@buildhost CentOS-5.4-x86_64-core]# pwd
/mnt/cdimages/CentOS-5.4-x86_64-core

[root@buildhost CentOS-5.4-x86_64-core]# rm repodata/* -f

[root@buildhost CentOS-5.4-x86_64-core]# cp /tmp/no-lang.comps.xml  repodata/comps.xml

Building the stuff

The RHEL3 based howto recommends to run the commands pkgorder, buildinstall, etc, on the build tree. This is no longer valid. A simple createrepo command takes care of everything.

Testing the RPMs for any dependency errors

It is a good idea to do a test install of your RPMs, in a test directory. This will point out any dependency problems.


[root@buildhost CentOS]# pwd
/mnt/cdimages/CentOS-5.4-x86_64-core/CentOS

[root@buildhost CentOS]# mkdir /tmp/testrpmdb

[root@buildhost CentOS]# rpm --initdb --dbpath /tmp/testrpmdb/ 

[root@buildhost CentOS]# rpm --test --dbpath /tmp/testrpmdb/ -ivh *.rpm
warning: acl-2.2.39-3.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID e8562897
Preparing...                ########################################### [100%]
[root@buildhost CentOS]#

[root@buildhost CentOS]# echo $?
0

[root@buildhost CentOS]# du -sh /tmp/testrpmdb/
640K    /tmp/testrpmdb/

Had there be a package in that directory, which would be dependent on some other packages, then you would definitely get errors, which you can fix. For an example, I copied the rpm for httpd in this build directory.

[root@buildhost CentOS]# cp /mnt/cdimages/CentOS-5.4-x86_64/CentOS/httpd-2.2.3-31.el5.centos.x86_64.rpm .

And then, I run the RPM test install again:

[root@buildhost CentOS]# rpm --test --dbpath /tmp/testrpmdb/ -ivh *.rpm
warning: acl-2.2.39-3.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID e8562897
error: Failed dependencies:
        libapr-1.so.0()(64bit) is needed by httpd-2.2.3-31.el5.centos.x86_64
        libaprutil-1.so.0()(64bit) is needed by httpd-2.2.3-31.el5.centos.x86_64
[root@buildhost CentOS]# 

As you can see, in the above example, either you remove the httpd RPM, or add libapr libraries too, to the CentOS directory inside your build directory. I am sure, you are intelligent enough to figure out the rest.

Note: Some documents on the net recommend that you copythe following in the CentOS directory in your build-tree, as it is suspected that they are needed for the anaconda installer to run properly. However, please note that I was able to create a CD "without" these RPMs , and was also able to install a VM using the the GUI of anaconda installer. These (not needed) packages are :

anaconda
anaconda-runtime
anaconda-help
anaconda-product
busybox
busybox-anaconda
memtest86+
fonts-xorg-base
xorg-x11
joe

createrepo

Now is the time to run createrepo. Before you run that, you need to get some information from the .discinfo file. It is included in the root directory of the CentOS/RHEL CD/DVD. If you did an rsync (or wget) to get these files from the source location, you should have it in the top level of your build-directory. Here is how it looks like:

[root@buildhost CentOS-5.4-x86_64-core]# pwd
/mnt/cdimages/CentOS-5.4-x86_64-core

[root@buildhost CentOS-5.4-x86_64-core]# cat .discinfo 
1254416275.453877
Final
x86_64
1,2,3,4,5,6,7
CentOS/base
/home/buildcentos/CENTOS/5.4/en/x86_64/CentOS
CentOS/pixmaps

Lets move forward.

[root@buildhost CentOS-5.4-x86_64-core]# export discinfo=$(head -1 /mnt/cdimages/CentOS-5.4-x86_64-core/.discinfo)

[root@buildhost CentOS-5.4-x86_64-core]# echo $discinfo
1254416275.453877

Now, run the createrepo command. Remember that we removed everything in the repodata directory and only placed our (very small/trimmed down), comps.xml in that directory.

[root@buildhost CentOS-5.4-x86_64-core]# createrepo -u "media://$discinfo" \ 
   -g repodata/comps.xml /mnt/cdimages/CentOS-5.4-x86_64-core

444/444 - CentOS/mailx-8.1.1-44.2.2.x86_64.rpm                                  
Saving Primary metadata
Saving file lists metadata
Saving other metadata
[root@buildhost CentOS-5.4-x86_64-core]#

Create Bootable ISO file

If the previous step did not give you any errors, you are now ready to create the actual bootable CD / ISO file out of this build-tree. (There is nothing worse then dead air!)

[root@buildhost CentOS-5.4-x86_64-core]# export isofilename=/mnt/cdimages/CentOS-5.4-x86_64-core.iso 

[root@buildhost CentOS-5.4-x86_64-core]# mkisofs -r -R -J -T -v  \
   -no-emul-boot -boot-load-size 4 -boot-info-table \
   -V "CentOS-5u4-core 64 bit" -p "Kamran"  -A "CentOS-5u4-core 64 bit-2010/07/02" \
   -b isolinux/isolinux.bin -c isolinux/boot.cat  -x "lost+found" \
   -o $isofilename /mnt/cdimages/CentOS-5.4-x86_64-core

INFO:   UTF-8 character encoding detected by locale settings.
        Assuming UTF-8 encoded filenames on source filesystem,
        use -input-charset to override.
mkisofs 2.01 (cpu-pc-linux-gnu)
Scanning /mnt/cdimages/CentOS-5.4-x86_64-core
Scanning /mnt/cdimages/CentOS-5.4-x86_64-core/images
. . . 

 95.65% done, estimate finish Sat Jul  3 20:20:09 2010
 97.60% done, estimate finish Sat Jul  3 20:20:10 2010
 99.56% done, estimate finish Sat Jul  3 20:20:08 2010
Total translation table size: 146726
Total rockridge attributes bytes: 62153
Total directory bytes: 98304
Path table size(bytes): 112
Done with: The File(s)                             Block(s)    255876
Writing:   Ending Padblock                         Start Block 255991
Done with: Ending Padblock                         Block(s)    150
Max brk space used c2000
256141 extents written (500 MB)
[root@buildhost CentOS-5.4-x86_64-core]#

Implant MD5SUM in the ISO file

The following will implant the ISO file's MD5 inside the ISO file itself.

[root@buildhost CentOS-5.4-x86_64-core]# implantisomd5 $isofilename
Inserting md5sum into iso image...
md5 = 48a3456517149f584ae81f793b933459
Inserting fragment md5sums into iso image...
fragmd5 = 6cae56578bb59bb8bd2289af35fa7c4f4468658de47f45e6d488be9b6b33
frags = 20
Setting supported flag to 0
[root@buildhost CentOS-5.4-x86_64-core]# 

Test the stuff you just build (the bootable ISO image)

Ok, now the CD is created, as a result of last (mkisofs) step. I tested it to create a VM , on my Fedora 11 KVM host. It worked without a problem, in both text and GUI mode. Here are a few screen shots!


File:Custom-cd-text-installer-small.png

File:Custom-cd-gui-installer-1-small.png File:Custom-cd-gui-installer-2-small.png File:Custom-cd-gui-installer-3-small.png

DONE

Here is some of the information from the newly installed VM. You can see that only ssh service is running, and the install size of the system is 701MB in total.

[kamran@kworkhorse cdimages]$ ssh root@192.168.122.61
The authenticity of host '192.168.122.61 (192.168.122.61)' can't be established.
RSA key fingerprint is a1:0e:0b:01:92:6a:69:14:e6:62:6d:83:6d:08:16:1d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.122.61' (RSA) to the list of known hosts.
root@192.168.122.61's password: 
Last login: Sat Jul  3 20:53:08 2010


[root@custom-cd ~]# netstat -antp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 :::22                       :::*                        LISTEN      1736/sshd           
tcp        0      0 ::ffff:192.168.122.61:22    ::ffff:192.168.122.1:48069  ESTABLISHED 2154/0              


[root@custom-cd ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1             2.7G  701M  1.9G  28% /
tmpfs                 250M     0  250M   0% /dev/shm
[root@custom-cd ~]# 

The first thing, which you might want to do on this newly installed system is update it full, to bring it upto the latest version of kernel and the distribution. The benefit of such small installation is that the update will pull relatively less amount of packages, compared to a larger install. If you are connected to the internet, this should complete in minutes. Later, whatever packages you want to add, they will be pulled from the updates repository (if you have it configured that way).

[root@custom-cd ~]#  yum -y update


Note: If you want, you can further remove extra packages from the newly installed system. You can use the following as a guide line: Packages which can be removed after Core installation .

Further reading

  1. http://cinq.com/2008/04/14/rebuilding-the-dvd-iso-for-centos-5-or-rhel-5/
  2. http://www.linuxjournal.com/article/6473?page=1,0
  3. http://www-oss.fnal.gov/projects/fermilinux/common/anaconda.comps.html
  4. http://fedoraproject.org/wiki/PackageMaintainers/CompsXml
  5. http://createrepo.baseurl.org/
  6. http://mirror.centos.org/centos/4/build/distro/build.sh.txt
Personal tools