Sunday, February 27, 2011

CPAN mirrors list now (also) in JSON format

The list of CPAN mirrors have for ages been published in the MIRRORED.BY file.  The format was good back when just parsing a text file was the common data exchange format (and I believe the "master file" was indeed the same file).


Now this file is output by a small program from another data source[1], so Henk Penning (the ever patient and hard working maintainer of the mirrors list) added a JSON export.  If you are making a tool that needs the CPAN mirrors list (and you are able to load a JSON file), it might be easier to use the new mirrors.json file.


 


 - ask


[1] The master data source is actually also a JSON file ...



Sunday, February 20, 2011

Now hosting the master mirror for CPAN

15 years ago Jarkko Hietaniemi started CPAN; now arguably the most important feature of Perl.  For all that time the canonical CPAN has been hosted at FUnet, with (now more than 600) mirrors around the world.  Having made much of my living using Perl I'm incredibly grateful to Jarkko and FUnet for having built and maintained this incredible resource for such a long time.


A few months ago Jarkko started passing the baton for looking after CPAN on to others in the community and as part of that we at perl.org are taking over the task of being the "master mirror" for CPAN.  Currently almost 500 of the CPAN mirrors are mirroring straight from FUnet which is an incredible resource drain for the master mirror.


The new system will be using the rrr tool for rapid mirroring to a set of "tier 1" mirrors so we more easily can scale to support anyone who wants to mirror CPAN.  File::Rsync::Mirror::Recent is already in use by some of the mirrors to get "instant" updates from PAUSEAndreas König (the inventor and long time maintainer of PAUSE) is working on some improvements to make it work better for cpan.org in general.


We're also working on getting things in place so the static pages on cpan.org more easily can be maintained and updated by the community.


If you are interested in helping with testing the mirroring process or anything else, please subscribe to the cpan-workers mailing list.


- Ask