www.centos.org - centos.org content

CentOS Mirroring HowTo

CentOS welcomes new mirror sites.

If you are considering setting up a public mirror site for CentOS please follow these guidelines to make sure that your mirror is consistent with the other mirror sites.

As a first step if you are not already subscribed, subscribe to the CentOS-mirror mailing list: http://lists.centos.org/mailman/listinfo/CentOS-mirror .

Public mirrors should syncronize 2-4 times per day and this should be run via cron. We highly recommend the use of lock files in your cron script so that you don\'t spawn multiple connections which is hard on our servers and on your mirror. If you don\'t know how to do this, please ask the list.

The best way to mirror the site inside of a cron job is:

rsync -aqzH --delete msync.centos.org::CentOS /path/to/local/mirror/root

EU users may wish to use eu-msync.centos.org::CentOS
US users may wish to use us-msync.centos.org::CentOS

(if you are using >= 2.6.4 version of rsync, you might want to specifiy the --delay-updates switch to the above line in your cron job)

Please email the list when you're all set up with your cron job and your initial sync is completed. We will add you to our list of mirror sites. Please provide all relevant URL's (ftp/http/rsync/?) as well as how often you are synchronizing the mirror network. Please include what city/state/country you are in and what your bandwidth cap is. Also please give a name and link to the sponsoring organization so we can give proper credit. If the best contact point for your mirror is not the address you are subscribed to the mailing list with, please provide the email address of your preferred contact as well.

Please alert the CentOS-mirror mailing list if anything about the server changes like location, available bandwidth, frequency of updates, etc.

We ask that Tier 2, personal, and company intranet mirror sites please pick a Tier 1 mirror rather than synchronizing directly from the master rsync server pool. We are not restricting the master rsync server pool in order to help new mirror sites get online expediently but will re-examine this issue if the master rsync server pool is overly used.

The comments are owned by the poster. We aren't responsible for their content.

Poster	Thread
karmapolis	Posted: 2006/5/5 23:27 Updated: 2006/5/5 23:27
Newbie Joined: 2006/5/5 From: Posts: 4	Re: Mirror HowTo I suggest that you add an option to the new mirrorlist.centos.org script so that corporate users are able to query specifically for rsync mirrors to build their intranet mirrors. This way, one could write a script to automatically choose from one server on that list, instead of having to rely on the only fixed one (the master rsync server) which is probably already burdened. For example, instead of... http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os ... one might try this one... http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&proto=rsync ... to get only rsync mirrors that are "nearby" (in GeoIP terms).

Poster	Thread
hughesjr	Posted: 2006/5/7 20:00 Updated: 2006/5/7 20:00
WebMaster Joined: 2004/12/5 From: Corpus Christi, Texas, USA Posts: 401 Online!	Re: Mirror HowTo we actually have a powerdns script that passes out geoip enabled IP address from our msync machines we several machines that are dedicated hosts and do rsyncs only ... so this has already been done, though not in the same way.

Poster	Thread
karmapolis	Posted: 2006/5/23 20:45 Updated: 2006/5/23 20:45
Newbie Joined: 2006/5/5 From: Posts: 4	Re: Mirror HowTo OK, but that method doesn't seem to be publicly available, right? I was specifically referring to private repositories, like the ones typically built in-house in all kinds of corporate environments. They use rsync, and currently there's no way for them to obtain a list of rsync mirrors automatically (none that I know, at least).

Poster	Thread
hughesjr	Posted: 2006/6/1 0:19 Updated: 2006/6/1 0:19
WebMaster Joined: 2004/12/5 From: Corpus Christi, Texas, USA Posts: 401 Online!	Re: Mirror HowTo That might be good, I'll look at it ... however, we already have a www page that shows that: http://www.centos.org/modules/tinycontent/index.php?id=13 You should be able to pick a fast, close mirror using that

Poster	Thread
mikebabcock	Posted: 2008/8/15 15:13 Updated: 2008/8/15 15:13
Newbie Joined: 2008/8/15 From: Posts: 1	Re: Mirror HowTo Have you looked at Fedora MirrorManager for handling mirrors? MirrorManager supports knowledge of private mirrors (including redirecting users from specific subnets to their site's published mirror) and several other features. http://fedoraproject.org/wiki/Infrastructure/MirrorManager

Poster	Thread
maxx	Posted: 2006/9/3 21:40 Updated: 2006/9/3 21:40
Newbie Joined: 2005/6/19 From: Posts: 4	Re: Mirror HowTo I am looking to set up an intranet mirror. Three questions... 1. Is there a way to restrict what is mirrored to Centos 4 i386/AMD64? (and how?) 2. Whether or not such restriction is possible, how much disk space should I allow? 3. How do I configure yum so that it uses my Intranet mirror? (I am a newbie with rsync and yum configuration--having typically cut-and-pasted out of howto's in any situation where I've needed them before.) Thanks.

Poster	Thread
jlar310	Posted: 2006/10/4 14:53 Updated: 2006/10/4 14:53
Newbie Joined: 2006/10/4 From: Posts: 1	Re: Mirror HowTo You have perhaps found your solution by now, but I'll post this for the benefit of all. You can do a partial mirror by using multiple rsync operations on the specific paths you wish to mirror. We have an intranet mirror of version 4 for i386 and x86_64. Here's my script which runs once a day. #!/bin/sh rsync="/usr/local/bin/rsync -aqHz --delete --delay-updates --bwlimit=256" # this should be auto-selected, but the centos mirror script doesn't # support selection by a specific protocol (i.e., rsync) mirror=mirror.stanford.edu::mirrors/centos #mirror=centos.cs.wisc.edu::centos #mirror=mirrors.usc.edu::centos ver=4 archlist="i386 x86_64" baselist="os updates" local=/usr/local/src/repos/centos for arch in $archlist do for base in $baselist do remote=$mirror/$ver/$base/$arch/ $rsync $remote $local/$ver/$base/$arch/ done done This particular setup requires about 5 GB. Configure yum by creating /etc/yum.repos.d/myrepos.repo as follows: `[base] name=CentOS-$releasever - Base baseurl=http://myserver.mydomain.com/centos/$releasever/os/$basearch/ # updates [update] name=CentOS-$releasever - Updates baseurl=http://myserver.mydomain.com/centos/$releasever/updates/$basearch/` Change the URL protocol and path as needed to fit your environment. I renamed all the other .repo files in /etc/yum.repos.d to .repo.disabled so that only my local server is accessed. But watch out as some updates will create new .repo files. Good luck! Jeff

Poster	Thread
lau-co	Posted: 2006/11/23 10:22 Updated: 2006/11/23 10:22
Newbie Joined: 2006/11/23 From: Posts: 1	Re: Mirror HowTo I'm trying to manage a local repository for a linux cluster with this script with restricting it to the v4 x86_64 architecture. It works fine except for one file : sysreport-1.3.15-8.noarch.rpm in updates base which is a symlink to the one in i386 repository. Since I didn't get the i386 architecture, the link is broken on my local mirror. I fix that by adding the 'L' option to the rsync command, but is this 'usual' to have such links across architectures ?

Poster	Thread
hughesjr	Posted: 2006/12/3 13:37 Updated: 2006/12/3 13:37
WebMaster Joined: 2004/12/5 From: Corpus Christi, Texas, USA Posts: 401 Online!	Re: Mirror HowTo normally a hard link though, not a symlink (for just that reason) we do want to link to save space ... but hardlinks should prevent issues. That one is now fixed.

Poster	Thread
keithyau	Posted: 2008/1/8 11:47 Updated: 2008/1/8 11:47
Newbie Joined: 2008/1/8 From: Posts: 1	Re: Mirror HowTo I come from HK and want to be one of centos 's mirror Would admin please telling me what i have to do ? The guideline in the webpage is only about 3 sentences and i'm not strong enough to understand it I have a host running 24 hours and using FreeBSD ~ I love CentOS, just i cannot make my hardware support it Regards, Keith

Poster	Thread
hkfnc	Posted: 2008/2/1 2:33 Updated: 2008/2/1 2:33
Newbie Joined: 2008/2/1 From: Posts: 1	Re: Mirror HowTo Dear Sir/Madam We also would like to become a public mirror but not really know how to do it. Please help. Thank you !

Poster	Thread
gigglesworth	Posted: 2009/9/8 23:41 Updated: 2009/9/8 23:41
Newbie Joined: 2005/5/23 From: Posts: 2	Symbolic links/hard links Can someone please explain how the symbolic links/hard links should work for the directories? What should I expect to see after rsync'ing the master? I ask this because I keep running into Mirrors which have a dead directory for older releases. http://mirror.example.com/centos/5/ and http://mirror.example.com/centos/2/ are empty directories, for example. I don't have the space (or time) to download the mirror myself, so I was hoping to get an explanation from someone else. Thanks, -= Stefan