Making a Local Copy of openSUSE Repository (Mirror Server)

If you have a stable internet access, unlimited bandwidth and want to share the repository around the network, why don’t you building a local copy of openSUSE repository ?
Mainly, there is 3 version of openSUSE repository, oss, non oss and Packman. Oss repository contains all of open source application and the non-oss contains free application without providing source code. Open source application such as Thunderbird, Kmail and OpenOffice could be found in the oss repository, while non-oss contains Flash-Player, Acrobat Reader, etc. The last one is Packman, which contains the 3rd party application such as Clamav, multimedia package, application with special purpose (e.g Kchm), etc.
We could build a local copy of openSUSE repository by using anonymous Rsync. I strongly recommended to use another mirror as base source rather than using main download.opensuse.org. It most because the primary source often busy and our Rsync process could increase the server traffic.
Here is how to create a local copy of openSUSE repository :
OSS & Non-OSS Package

  1. Install Rsync package, by using YAST | Software | Software Management or from konsole with this command:
  2. [code language=’cpp’]zypper in rsync [/code]

  3. Check the mirror content with anonymous Rsync
  4. [code language=’cpp’]rsync ftp5.gwdg.de:: [/code]

  5. Check the content of openSUSE folder
  6. [code language=’cpp’]rsync ftp5.gwdg.de::pub/opensuse/ [/code]

  7. Making a local copy (Adjust the target folder. I’m currently using /srv/www/htdocs. Beware with the trailing slash “/” (ex : ftp5.gwdg.de::pub/opensuse/distribution/) at the end of folder on the base source URL. Adding the “/” at the end of base source means that we want to copied the CONTENT of the specified folder. If we remove the trailing slash means that we want to copy the folder)
  8. [code language=’cpp’]rsync -av –delete-after ftp5.gwdg.de::pub/opensuse/distribution /srv/www/htdocs [/code]

Note : –delete-after means that we want to remove all of file and folder if it was not found in the source and the process will be executed after finishing copy process. Used the exclude prefix if we want to resticted some folder from being copy. Ex : If i wanto to exclude the debug, src-non-oss and src-oss from being copied, I write a text file with the contens :
debug/
src-non-oss/
src-oss/

and then running the 8th step with this command :
[code language=’cpp’]rsync -av –delete –exclude=exclude.txt ftp5.gwdg.de::pub/opensuse/distribution/ /srv/www/htdocs [/code]
Packman Package

I’m using packman repository on the packman.mirrors.skynet.be. Please feel free to change the server with your nearest mirror.
[code language=’cpp’]rsync -av –delete packman.mirrors.skynet.be::packman/suse/ /srv/www/htdocs/packman/[/code]
Add the command into schedule cron job for regular update.

16 thoughts on “Making a Local Copy of openSUSE Repository (Mirror Server)

  1. Shouldn’t your above step with the excludes:
    “rsync -av –delete –exclude=exclude.txt ftp5.gwdg.de::pub/opensuse/distribution/ /srv/www/htdocs”
    Read:
    1.
    rsync -av –delete –exclude-from=exclude.txt ftp5.gwdg.de::pub/opensuse/distribution/ /srv/www/htdocs

  2. Shouldn’t your above step with the excludes:
    “rsync -av –delete –exclude=exclude.txt ftp5.gwdg.de::pub/opensuse/distribution/ /srv/www/htdocs”
    Read:
    1.
    rsync -av –delete –exclude-from=exclude.txt ftp5.gwdg.de::pub/opensuse/distribution/ /srv/www/htdocs

  3. This is a neat idea, but what I’d really like to see is a caching proxy setup where you set a local location as your repository and it proxies data from say the oss repository. When a user asks for a file it checks to see if it exists locally, if it does, it gives the user the file. Otherwise it downloads the file from the mirror and streams it to the user on the fly. After it is done downloading, it keeps it locally, as presumably other users on the network may want the file as well. This sort of proxy would scale much better than mirroring the whole repository, especially for people in lower bandwidth situations (like me). I don’t know if anything like this is available yet.

  4. This is a neat idea, but what I’d really like to see is a caching proxy setup where you set a local location as your repository and it proxies data from say the oss repository. When a user asks for a file it checks to see if it exists locally, if it does, it gives the user the file. Otherwise it downloads the file from the mirror and streams it to the user on the fly. After it is done downloading, it keeps it locally, as presumably other users on the network may want the file as well. This sort of proxy would scale much better than mirroring the whole repository, especially for people in lower bandwidth situations (like me). I don’t know if anything like this is available yet.

  5. Thanks man was really helpful
    For me this worked out
    rsync -av –delete –exclude-from=exclude.txt ftp5.gwdg.de::pub/opensuse/distribution/ /srv/www/htdocs
    But it was properly explained in your post.

  6. Thanks man was really helpful
    For me this worked out
    rsync -av –delete –exclude-from=exclude.txt ftp5.gwdg.de::pub/opensuse/distribution/ /srv/www/htdocs
    But it was properly explained in your post.

  7. after running this command
    rsync -av –delete-after ftp5.gwdg.de::pub/opensuse/distribution /srv/www/htdocs
    it automatically downloads repositories for 11.0 but what if i want to download 11.2 repositories?

Leave a Reply

Your email address will not be published. Required fields are marked *