Build Your Own RHEL, sorta!

A 5 star hangout for overworked and underpaid system admins.
ancwd
Posts: 8
Joined: 2011/02/20 19:07:22

Build Your Own RHEL, sorta!

Post by ancwd » 2011/02/20 19:18:13

Hey everyone,

if you're following the some of the posts in the social forum and devel mailing list, i'm sure you've come across something like this "build your own rhel" well in this post i'll try to explain how do that, sorta. at least keeping it up to date ;)

I won't say that this is the best approach, most efficient one or even the right way of doing it. but nonetheless it's a start. besides i'm learning as i'm going, and i'm sharing it with whoever is interested.

why RHEL 5.6 and not RHEL 6.0? I don't have centos or rhel on it's own machine, i always use virtualbox, unfortunatley i can't get RHEL 6.0 to work with virtualbox due to a virtualbox bug, but RHEL 5.6 works fine.

Ok let's go.

### Overview of what's up ahead ###

1) Installing RHEL 5.6 from redhat.com
2) Setting up our very own repo based on the DVD rpms and Source RPMS
3) Setting up an Updates repo that we will populate with our built rpms
3) Create a script that checks if there are updates available for our installtion.
a) Download only the needed and latest source rpms, depending on what's already installed.
b) rebuild the downloaded source rpms and add the resulting binary rpms to our Updates repo
4) do yum update :)


Note: to rebuild the rpms i'll be using rpmbuild, since i'm clueless about mock/plague, but if anyone knows how to set those up, please contribute.

Note: i won't be covering how to setup anaconda, create a distro installation dvd or anything like that. again since i don't know how, but if you know how to do that as well please chip in :)


### Installation ###
Let's start, go to redhat.com, sign up for the 30-day trial, and download rhel and the source dvd.
I downloaded rhel-server-5.6-i386-dvd.iso and rhel-server-5.6-source-dvd.iso

Install rhel, it's the same as centos except when it asks for an installation number just select skip.

When you reach the package selection phase, select "Software Development" and "Web server", we'll need both of those things.
Optional: Select Customize Now and add anything else you might need.

Note: after firstboot i changed the SELinux settings to Permissive, you don't have to do that if you're efficient with SELinux

Again when it asks if you would like to register your system? I selected register at a later time.

moving on ....

### Setting up the repos and hosting them via apache ###

Let's create some directories in /var/www to host our repos (hint: if you're having problems with apache and the repos, check that selinux is setup correctly)

$> mkdir -p /var/www/html/redhat/base/5Server/i386
$> mkdir -p /var/www/html/redhat/base/5Server/rhel-source
$> mkdir -p /var/www/html/redhat/updates/5Server/i386



Copy the binary rpms (*.rpm) on the DVD to the i386 folder.
$> cp -v /media/RHEL_5.6\ i386\ DVD/Server/*.rpm /var/www/html/redhat/base/5Server/i386/
$> cp -v -r /media/RHEL_5.6\ i386\ DVD/Server/repodata/ /var/www/html/redhat/base/5Server/i386/


Now put in the source dvd and copy the Source rpms (*.src.rpm) to the rhel-source folder.
$> cp -v /media/RHEL_5.6\ Source/SRPMS/*.src.rpm /var/www/html/redhat/base/5Server/rhel-source
$> cp -v -r /media/RHEL_5.6\ Source/SRPMS/repodata/ /var/www/html/redhat/base/5Server/rhel-source



As root, create a file /etc/yum.repos.d/rhel-base.repo
$> nano /etc/yum.repos.d/rhel-base.repo

Copy and paste the following:

[rhel-base]
name=Red Hat Enterprise Linux $releasever - $basearch - Base
baseurl=http://localhost/redhat/base/5Server/i386/
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

[rhel-updates]
name=Red Hat Enterprise Linux $releasever - $basearch - Updates
baseurl=http://localhost/redhat/updates/5Server/i386/
enabled=0
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

[rhel-base-source]
name=Red Hat Enterprise Linux $releasever - Base Source
baseurl=http://localhost/redhat/base/5Server/rhel-source/
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release



Let's start apache and see if we can access the repos in a web browser:

$> chkconfig httpd on
$> /sbin/service httpd start

Open firefox and in the address bar type in http://localhost/redhat/

Now let's see if we can install packages from this repo

$> yum install createrepo yum-utils


You should see something like this:

[root@rhel56 ~]# yum install createrepo yum-utils
Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
rhel-base | 1.3 kB 00:00
rhel-base/primary | 777 kB 00:00
rhel-base 2428/2428
rhel-base-source | 951 B 00:00
rhel-base-source/primary | 327 kB 00:00
rhel-base-source 1231/1231
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package createrepo.noarch 0:0.4.11-3.el5 set to be updated
---> Package yum-utils.noarch 0:1.1.16-13.el5_4.1 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================
Package Arch Version Repository Size
====================================================================================================
Installing:
createrepo noarch 0.4.11-3.el5 rhel-base 59 k
yum-utils noarch 1.1.16-13.el5_4.1 rhel-base 69 k

Transaction Summary
====================================================================================================
Install 2 Package(s)
Upgrade 0 Package(s)

Total download size: 129 k
Is this ok [y/N]: y
Downloading Packages:
(1/2): yum-utils-1.1.16-13.el5_4.1.noarch.rpm | 69 kB 00:00
(2/2): createrepo-0.4.11-3.el5.noarch.rpm | 59 kB 00:00
----------------------------------------------------------------------------------------------------
Total 1.3 MB/s | 129 kB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : createrepo 1/2
Installing : yum-utils 2/2

Installed:
createrepo.noarch 0:0.4.11-3.el5 yum-utils.noarch 0:1.1.16-13.el5_4.1

Complete!
[root@rhel56 ~]#


Since we downloaded createrepo, let's setup the updates repo and enable it.

$> createrepo /var/www/html/redhat/updates/5Server/i386

now edit the /etc/yum.repos.d/rhel-base.repo file and change enabled=0 to enabled=1 for [rhel-updates]

$> nano /etc/yum.repos.d/rhel-base.repo
$> yum update
Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
rhel-base | 1.3 kB 00:00
rhel-base-source | 951 B 00:00
rhel-updates | 951 B 00:00
rhel-updates/primary | 199 B 00:00
Skipping security plugin, no data
Setting up Update Process
No Packages marked for Update
$>


moving on ...

### Getting the latest sources ###

(note: you can create a bash script to do this in one go, but here i'm describing it line by line)

If you have a good internet connection and lot's and lot's of free space, then you can just rsync the SRPMS folder from ftp.redhat.com and get it over with, but for me, my internet speed is very slow and i'm quickly running out of free space.

So i need to update my distro, but I don't want to download older sources than the ones I have, and I don't want to download the sources of packages that I don't even have installed.

The secret of doing that is a tool called reposync that is part of yum-utils. unfortunatley the version shipped with rhel doesn't support the --source flag that allows the download of source rpms. so let's see what we can do.

$> rpm -qi yum-utils
will give us the link http://linux.duke.edu/yum/download/yum-utils/
let's go there and download the latest version which is this --> http://yum.baseurl.org/download/yum-utils/yum-utils-1.1.30.tar.gz

Download it and extract it:
$> wget http://yum.baseurl.org/download/yum-utils/yum-utils-1.1.30.tar.gz
$> tar -xzf yum-utils-1.1.30.tar.gz


Let's run it and see if it works fine.

$> /root/yum-utils-1.1.30/reposync.py --repoid=rhel-source --source -u -n -p /var/www/html/redhat/base/5Server/ | tee ancwd_reposync.txt

The parameters are
--repoid that's the repo we want to sync, rhel-source is defined in /etc/yum.repos.d/rhel-source.repo (note that for our source repo, i gave it the exact name)
-u Don't download anything just show the links
-n Don't show older versions only newer ones
-p where our version of the rhel-source folder is located.
| tee reposync.txt directs the output to the text file reposync.txt so we can manipulate it.


Ok now let's decide on what to download. to do this we'll make a list of what is available and a list of what is installed and compare them using the comm command.

# Get a list of available updates
$> cat ancwd_reposync.txt | grep -v 'Skipping existing' | sed 's/^.*\///' | sed 's/-[0-9].*$//' | sort | tee ancwd_available.txt

# Get a list of the installed packages
$> rpm -qa | sed 's/-[0-9].*$//' | sort | tee ancwd_installed.txt

# Compare the two and find matches
$> comm -1 -2 ancwd_available.txt ancwd_installed.txt | sed 's/$/-/g' | tee ancwd_updates.txt

# Generate download script
$> cat ancwd_reposync.txt | grep -v 'Skipping existing' | grep -f "ancwd_updates.txt" | sed 's/^/wget -c /' | tee ancwd_download.sh


After running those commands in that order we should have a file called ancwd_download.sh that lists exactly the files that we need to download:
in my case they are:

wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/coreutils-5.97-23.el5_6.4.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/cups-1.3.7-26.el5_6.1.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/curl-7.15.5-9.el5_6.2.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/hplip-1.6.7-6.el5_6.1.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/java-1.6.0-openjdk-1.6.0.0-1.20.b17.el5.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/kernel-2.6.18-238.1.1.el5.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/kexec-tools-1.102pre-126.el5_6.4.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/libuser-0.54.7-2.1.el5_5.2.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/m2crypto-0.16-6.1.el5_5.1.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/mysql-5.0.77-4.el5_5.5.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/pango-1.14.9-8.el5_6.2.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/psmisc-22.2-7.el5_6.2.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/quota-3.13-1.2.5.el5.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/subversion-1.6.11-7.el5_6.1.src.rpm
wget -c ftp://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/tzdata-2010o-1.el5.src.rpm

Note: if the ftp.redhat.com is not working for you, just relace ftp://ftp.redhat.com/ with another mirror like http://mirrors.kernel.org/

execute ancwd_download.sh and download the files.
It is preferred if you download the files in the /var/www/html/redhat/base/5Server/rhel-source folder so they don't appear next time you run it.

### Rebuilding Regular Packages ###
For this example i'll rebuild the curl package since it needs updating and should be straight forward.

let's start by setting up the build enviroment:
$> mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
$> echo '%_topdir %(echo $HOME)/rpmbuild' > ~/.rpmmacros

Now let's rebuild the package.

$> rpmbuild --rebuild curl-7.15.5-9.el5_6.2.src.rpm

If all goes well the new rpms are located in ~/rpmbuild/RPMS/i386/
$> ls ~/rpmbuild/RPMS/i386
curl-7.15.5-9.2.i386.rpm curl-debuginfo-7.15.5-9.2.i386.rpm curl-devel-7.15.5-9.2.i386.rpm

we don't need the debuginfo package so you can just delete that.

If the build fails, it might give you clues to what's wrong, such as missing a certain package to complete the build, simply yum install that package.


### Updating the system using YUM ###

copy curl-7.15.5-9.2.i386.rpm and curl-devel-7.15.5-9.2.i386.rpm to our updates repo folder

$> mv curl-7.15.5-9.el5_6.2.src.rpm /var/www/html/redhat/updates/5Server/i386/
$> mv curl-devel-7.15.5-9.2.i386.rpm /var/www/html/redhat/updates/5Server/i386/


Update the repo using createrepo
$> createrepo /var/www/html/redhat/updates/5Server/i386/

I'm not sure why it's not working right away, but do a yum clean all followed by yum update.
curl should be available now for update!!

$> yum clean all
$> yum update

[root@rhel56 ~]# yum update
Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
rhel-base | 1.3 kB 00:00
rhel-base/primary | 777 kB 00:00
rhel-base 2428/2428
rhel-base-source | 951 B 00:00
rhel-base-source/primary | 327 kB 00:00
rhel-base-source 1231/1231
rhel-updates | 951 B 00:00
rhel-updates/primary | 1.3 kB 00:00
rhel-updates 2/2
Skipping security plugin, no data
Setting up Update Process
Resolving Dependencies
Skipping security plugin, no data
--> Running transaction check
---> Package curl.i386 0:7.15.5-9.2 set to be updated
---> Package curl-devel.i386 0:7.15.5-9.2 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================
Package Arch Version Repository Size
====================================================================================================
Updating:
curl i386 7.15.5-9.2 rhel-updates 267 k
curl-devel i386 7.15.5-9.2 rhel-updates 311 k

Transaction Summary
====================================================================================================
Install 0 Package(s)
Upgrade 2 Package(s)

Total download size: 578 k
Is this ok [y/N]: y
Downloading Packages:
(1/2): curl-7.15.5-9.2.i386.rpm | 267 kB 00:00
(2/2): curl-devel-7.15.5-9.2.i386.rpm | 311 kB 00:00
----------------------------------------------------------------------------------------------------
Total 6.3 MB/s | 578 kB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : curl 1/4
Updating : curl-devel 2/4
Cleanup : curl 3/4
Cleanup : curl-devel 4/4

Updated:
curl.i386 0:7.15.5-9.2 curl-devel.i386 0:7.15.5-9.2

Complete!


### Rebuilding the almighty Kernel and complex programs (gcc, binutils, kernel modules) ###

I haven't finished this step yet, but once i figure it out i'll update this post.
For now check the following links:

http://www.howtoforge.com/kernel_compilation_centos
http://wiki.centos.org/HowTos/Custom_Kernel

http://www.linuxfromscratch.org/lfs/downloads/stable/
http://tldp.org/LDP/Pocket-Linux-Guide/html/

User avatar
AlanBartlett
Forum Moderator
Posts: 9345
Joined: 2007/10/22 11:30:09
Location: ~/Earth/UK/England/Suffolk
Contact:

Re: Build Your Own RHEL, sorta!

Post by AlanBartlett » 2011/02/20 20:51:27

There's many ways to do this and it is obvious you are having fun in the process.

With regard to the four links at the end of your post, the first one is [b]VeryWrong[/b](tm) and is severely frowned upon. :-x Upon reading the introduction of the second link, you will see the reason why. ;-)

As long as you keep the end result for your own personal use and do not provide any binaries to other persons, then [url=http://wiki.centos.org/FAQ/General?highlight=%28TUV%29#head-d29a2b7e61ffc544973098f9dd49fe4663efba50]TUV[/url] will leave you in peace.

assassin4700
Posts: 11
Joined: 2011/02/10 06:45:00

Re: Build Your Own RHEL, sorta!

Post by assassin4700 » 2011/02/20 23:26:17

What happens when Red Hat only allows paying customers to get an 30 day eval? They could do this you know!

Secondly, what happens when only verifiable email addresses are allowed to connect to RHN? No Yahoo, GMAIL, Hotmail, etc...

I can see a post like this pissing Red Hat off where they will continue to make it difficult for CentOS.

You should think about this the next time you post such nonsense.

gerdesas
Posts: 32
Joined: 2009/04/21 18:03:13
Location: Manchester, TN USA
Contact:

Re: Build Your Own RHEL, sorta!

Post by gerdesas » 2011/02/21 00:04:42

[quote]
assassin4700 wrote:
What happens when Red Hat only allows paying customers to get an 30 day eval? They could do this you know!

Secondly, what happens when only verifiable email addresses are allowed to connect to RHN? No Yahoo, GMAIL, Hotmail, etc...

I can see a post like this pissing Red Hat off where they will continue to make it difficult for CentOS.

You should think about this the next time you post such nonsense.[/quote]

Sigh...

There is no "nonsense" here at all. People are able to rebuild RHEL to their heart's content. If the trademark material is elided from their results they can even distribute it as they see fit. This is all above-board.

There is nothing to be gained by Redhat restricting the 30-day trials to their existing customers and in fact it's extremely poor business sense to do so; Redhat gains paying customers by giving people a 30-day trial period to see if the distro is a good fit for their needs.

I don't see the point to your comments about "verifiable email addresses". Are you referring to continuing to get updates past the 30-day period by using multiple e-mail addresses?

As far as making things difficult for CentOS... Do you have proof that Redhat has in any way _intentionally_ made things difficult for CentOS or any other project that is rebuilding EL source? This is pure conjecture on your part, and for the record I think you're completely wrong.

User avatar
AlanBartlett
Forum Moderator
Posts: 9345
Joined: 2007/10/22 11:30:09
Location: ~/Earth/UK/England/Suffolk
Contact:

Re: Build Your Own RHEL, sorta!

Post by AlanBartlett » 2011/02/21 02:43:19

[b]assassin4700[/b] and [b]gerdesas[/b],

I would appreciate it if you do not turn this thread into a flame-war or personal battle-ground. :-x

Yes, it is the [i]CentOS-Social[/i] forum where almost anything can be discussed -- hence the reason why I have allowed the OP's posting to stand intact. ;-)

I recognise both of you by ID and know that you both have very strong opinions on various subjects -- often orthogonal to each other. That diversity of ideas and opinions is good. The world would be a very boring place, otherwise. (Imagine the [i]Redmond OS[/i], [i]'doze[/i], everywhere . . . :cry: )

If you feel that you must continue in "horns locked" mode, please do it either by PM or by e-mail. :roll:

I've said my piece so, with my moderating wand to hand, I shall return to my :pint:

gerdesas
Posts: 32
Joined: 2009/04/21 18:03:13
Location: Manchester, TN USA
Contact:

Re: Build Your Own RHEL, sorta!

Post by gerdesas » 2011/02/21 05:43:34

[quote]
AlanBartlett wrote:
[b]assassin4700[/b] and [b]gerdesas[/b],

I would appreciate it if you do not turn this thread into a flame-war or personal battle-ground. :-x
[/quote]

And I'd appreciate you not jumping to conclusions based on a post that was in no way negative.

User avatar
AlanBartlett
Forum Moderator
Posts: 9345
Joined: 2007/10/22 11:30:09
Location: ~/Earth/UK/England/Suffolk
Contact:

Re: Build Your Own RHEL, sorta!

Post by AlanBartlett » 2011/02/21 20:57:05

[quote]
And I'd appreciate you not jumping to conclusions based on a post that was in no way negative.
[/quote]
I shall remind you to please abide by perfectly reasonable [url=https://www.centos.org/modules/newbb/viewtopic.php?topic_id=14274&forum=47]forum[/url] guidelines. Point #2 is applicable to you.

You have already received one private e-mail from the moderating team with regards to your posts to forum threads. I was giving you some advice . . . now it is a public warning.

User avatar
toracat
Site Admin
Posts: 7518
Joined: 2006/09/03 16:37:24
Location: California, US
Contact:

Build Your Own RHEL, sorta!

Post by toracat » 2011/02/21 22:16:36

A note about what to do when you do not agree with the forum moderators or their actions is also found in the guidelines.

About the CentOS Social forum ... we started this forum some time ago when we realized there was a need for a place where forum users can freely talk on non-CentOS related issues, exchange ideas, so on and so forth. Now, recent 'not-so-pretty' threads are attracting people who come to the forums just to post to the Social forums. We would rather want to see them spend their time to actually [b]help[/b] others by providing answers to the [b]CentOS-related[/b] [technical] questions.

Perhaps, we should set up a rule that goes, "You must help at least one person in non-Social forums before you can post a message in the Social section". :-o <= joke, it's a joke. :-P

ancwd
Posts: 8
Joined: 2011/02/20 19:07:22

Re: Build Your Own RHEL, sorta!

Post by ancwd » 2011/02/21 23:41:46

ok i'm working on the steps needed to rebuild using mock.
i'm using the epel version and i'm trying to force it to only use the local repo,
there's also the centos-extras version, it didn't work as expected on my first try, i'm getting back to it later.
i'll update the post once i sort all of the details out.

later

User avatar
AlanBartlett
Forum Moderator
Posts: 9345
Joined: 2007/10/22 11:30:09
Location: ~/Earth/UK/England/Suffolk
Contact:

Re: Build Your Own RHEL, sorta!

Post by AlanBartlett » 2011/02/22 03:43:40

[quote]
i'll update the post once i sort all of the details out.
[/quote]
Please do. It's an interesting read seeing how your experiments turn out.

With regard to building the kernel, it really depends upon for which architecture you are building.

If 32-bit, you will need to perform three separate builds:

With [b]--target noarch[/b] to produce the documentation package.
With [b]--target i386[/b] to produce the headers package.
With [b]--target i686[/b] to produce the kernels.

If 64-bit, only two separate builds will be required:

With [b]--target noarch[/b], for the documentation.
With [b]--target x86_64[/b], for both the headers and the kernels.

If you do not wish to build, for example, the debug or xen kernels there are various options you can use on the [i]rpmbuild[/i] command line. They will take the format such as [b]--with phoo --without bah[/b]. Both reading the beginning of the kernel spec file and the [i]CentOS[/i] wiki page, [url=http://wiki.centos.org/HowTos/Custom_Kernel]I Need to Build a Custom Kernel[/url], will help you to make the correct choice.

Post Reply