How to Create a Bioconductor Mirror Site

The Bioconductor package repositories may be mirrored with rsync. If you would like to become a mirror for package and data package repositories, please use the commands below.

Note that mirroring can also be accomplished with the udr command. If you have installed udr locally, you can just prepend udr to the rsync commands below. This should result in a massive increase in file transfer speed. The mirroring instructions at UCSC provide more details, including information on firewall ports that may need to be opened at your instution in order for udr to work.

BioC release repos

If you want to mirror the current Bioconductor release version (currently 3.0), please use the following commands:

Directory structure

Pick a destination directory where files will be mirrored. Let's say this will be in /dest. This directory should be served by your web server. Under that you'll need a directory called packages. This directory must be present as it is part of the structure of a Bioconductor repository. Underneath packages should be a directory corresponding to the versions of Bioconductor that you will host. The current release version is 3.0 and the current devel version is 3.1. We recommend you use symlinks called release and devel that always point to the current release and devel versions; this way you will never have to change your rsync commands. But you should change the symlink targets with every Bioconductor release (see the release schedule for exact dates).

The following commands will create the directory structure you'll need (remember that /dest is just an example of the destination directory you could use; you can put this directory anywhere on your system where there is enough free space).

mkdir -p /dest/packages
mkdir /dest/packages/3.0 # current release
mkdir /dest/packages/3.1 # current devel
ln -s /dest/packages/3.0 /dest/packages/release # change these links
ln -s /dest/packages/3.0 /dest/packages/devel   # every 6 months (with Bioc release)

All Bioconductor release repos (RECOMMENDED)

rsync -zrtlv --delete master.bioconductor.org::release /dest/packages/release

Bioconductor release Software repo

rsync -zrtlv --delete master.bioconductor.org::release/bioc /dest/packages/release /bioc

Bioconductor release Data repos

rsync -zrtlv --delete master.bioconductor.org::release/data /dest/packages/release/data

Bioconductor release Extra repo

rsync -zrtlv --delete master.bioconductor.org::release/extra /dest/packages/release/extra

BioC devel repos

If you want to mirror the Bioconductor devel repos (currently 3.1), please use the following commands:

All Bioconductor 3.1 repos (RECOMMENDED)

rsync -zrtlv --delete master.bioconductor.org::devel /dest/packages/devel

Bioconductor release Software repo

rsync -zrtlv --delete master.bioconductor.org::devel/bioc /dest/packages/devel/bioc

Bioconductor devel Data repos

rsync -zrtlv --delete master.bioconductor.org::devel/data /dest/packages/devel/data

Bioconductor devel Extra repo

rsync -zrtlv --delete master.bioconductor.org::devel/extra /dest/packages/devel/extra

Additional information

Make sure the directory above packages is served by a web server.

Bioconductor is big (> 188GB for BioC 3.1). Please check the size of what will be transferred with e.g. rsync -avn master.bioconductor.org::release and make sure you have enough room on your local disk before you start.

It is recommended that package repositories be synced once per day, scheduled with cron.

Begin using your new local repository, by making it accessible on your webserver. See the "contriburl" option to install.packages() (utils) for more information.

Finally, contact us if you would like to have your mirror listed on our mirror page and in R's chooseBioCmirror() function.

The Bioconductor master package repositories reside at Fred Hutchinson Cancer Research Center in Seattle, WA, USA.

Fred Hutchinson Cancer Research Center