Wednesday, May 16, 2018

The end of an era: Saying goodbye to search.cpan.org

It's with sad hearts that we are announcing that search.cpan.org will be retired on the 25th of June 2018.

Graham Barr originally wrote the site nearly 20 years ago -- it first went live in early 1999 -- and it quickly became an invaluable resource for Perl developers around the world.

The ability to search CPAN (Comprehensive Perl Archive Network) and read Perl module documentation online helped spark many developers interest in Perl and helped to build the Perl community.

The site was originally hosted by Washington University in St. Louis on a single Solaris box. For his work on search.cpan.org, Graham won a White Camel award in 2002.

The site was later moved to the Perl NOC. US mirrors were hosted by YellowBot and Phyber and a European mirror has been hosted by digital craftsmen for the last 10 years. These amazing people and companies helped make the site a success.

In recent years maintenance has become a burden. Most of the site is running 2005 era Perl code.   Luckily, there is now a viable alternative: MetaCPAN.org. The MetaCPAN team has been getting ready for the transition and is nearly ready to take over.

Shortly, a link will be added to all pages on search.cpan.org to inform users of the upcoming change.  After about a month, all traffic will be redirected to the equivalent MetaCPAN page.

We would like to thank all who have assisted with this project.  An extra special thanks to Graham for his hard work and unwavering support of search.cpan.org for all these years.  To search.cpan.org -- we will miss you.

Update #1 (2018-05-19):  There's some confusion around whether or not existing search.cpan.org URLs will continue to work.  They will!  The vast majority of of them will be transparently served by MetaCPAN pages.  You do not need to update any links right now.

40 comments:

  1. Thanks to Graham for starting such a useful community resource and to everyone else who has been involved in running it over the years.

    It's hard to recall a time before the existence of CPAN search - it's now such a vital part of the Perl ecosystem.

    ReplyDelete
  2. I hope there will be a redirect from search to metacpan. I still think of search as canonical.

    Thanks, Graham.

    ReplyDelete
    Replies
    1. Yeah, it's good that metacpan.org has features in place to provide replacements for all search.cpan.org URLs, like this post says.

      It still seems like a huge shame that use.perl.org disappeared without providing any kind of continuity for its links. I wonder what happened there!

      Delete
    2. From the post:

      "After about a month, all traffic will be redirected to the equivalent MetaCPAN page."

      I assume it'll be a 301 so Google will eventually update all of the entries in their index.

      Delete
  3. I echo Dave Cross's gratitude to Graham Barr and the Perl NOC for search.cpan.org.

    I am sad that it is going away. I've always preferred it to metacpan.org.

    An important question: What provision will be made for other internet resources which use search.cpan.org as their canonical cpan-search tool?

    I speak of FreeBSD ports in particular. Each FreeBSD port has a 'pkg-descr' file which, if it's a port of a CPAN distribution, has a line like this:

    #####
    WWW: http://search.cpan.org/dist/CPAN-Testers-Common-Client/
    #####

    There are thousands of such ports. What will the update plan for that be?

    Thank you very much.
    Jim Keenan

    ReplyDelete
    Replies
    1. Yes. Hopefully the redirects will take those URLs and redirect them intelligently.

      Delete
    2. The post says that auotmatic redirections will be put in place.

      But this does raise an interesting issue. We should get in touch with all the groups who package CPAN modules in that way and ask them to update their packages.

      Delete
    3. I'll raise the issue on the FreeBSD ports mailing list. We may be able to bulk-find-replace all of them.

      Delete
    4. FreeBSD fixed this for all Perl ports a couple days ago.
      https://svnweb.freebsd.org/ports?view=revision&revision=470993

      Delete
  4. That's a shame. I prefer the clean interface of search.cpan.org over the metacpan site.

    Where's the source code? May be kinda fun to play around with.

    ReplyDelete
    Replies
    1. https://github.com/metacpan/metacpan-web for the interface, https://github.com/metacpan/metacpan-api for the search/index backend

      Delete
    2. If you meant the source code of cpansearch, it has never been published.

      Delete
    3. Never been published? So nobody can help with maintenance? Why?

      Delete
    4. @unknown i don’t know Graham’s reasoning, but in my experience your question says that you vastly overestimate the chance people actually help and underestimate the effort it is to receive help (half baked patches, dealing with other people’s bugs). The cpan.org site is on GitHub and has years old bugs that nobody is helping with.

      Delete
    5. It should be noted that not everything published on the Internet is always in a state that you can publish the source code...

      Gabor Szabo made a MetaCPAN-SCO project on Github. It was an open source search.cpan.org clone that uses the MetaCPAN API as a back-end.

      https://github.com/szabgab/MetaCPAN-SCO

      Delete
    6. @unknown, there's an old term in Perl, the 'milliBarr' which applies to Graham's particular style. I patched it a number of times, but there was one problem I couldn't quite put my finger on so, I asked JHI, then the P5 Pumpking, to have a look and even he was bewildered. I wonder if his #WTF comment is still in there. :)

      Delete
    7. @Ask Bjørn Hansen, at https://github.com/perlorg/cpanorg there are no issues. Where can I find the "years old bugs"?

      Delete
    8. @Unknown - I added an issue at https://github.com/perlorg/cpanorg/issues/38 -- the original report was in various places I guess, ugh. Thank you.

      Delete
  5. The links should all keep working. The MetaCPAN folks will consider it a bug if they don’t (I think there are a few exceptions for things that don’t have an equivalent page on metacpan).

    ReplyDelete
  6. Correct, almost all existing URLs are supported (including the s.c.o api) there are a few metacpan does not have an equivalent for (browsing authors alphabetically for example) but we didn't feel were going to be missed.

    https://github.com/metacpan/sco-redirect is the redirection code.

    ReplyDelete
  7. I find the current wording on http://search.cpan.org a bit harsh and reminiscent of "Perl is dead" - esp. if people do not click through to the NOC posting.

    Maybe somebody can change it to something like (taken from the NOC announcement):

    --- START ---

    It's with sad hearts that we are announcing that search.cpan.org will be retired on the 25th of June 2018.

    Luckily, there is now a viable alternative: MetaCPAN.org. The MetaCPAN team has been getting ready for the transition and is nearly ready to take over.

    Shortly, a link will be added to all pages on search.cpan.org to inform users of the upcoming change. After about a month, all traffic will be redirected to the equivalent MetaCPAN page.

    We would like to thank all who have assisted with this project. An extra special thanks to Graham for his hard work and unwavering support of search.cpan.org for all these years. To search.cpan.org -- we will miss you.

    --- END ---

    See also
    https://twitter.com/domm_favors_irc/status/997372975946100736

    Greetings,
    domm

    PS: And of course a lot of thanks to all people involved in setting up / running search.cpan.org!!

    ReplyDelete
  8. Would be nice if a redirect was not required, i.e. make search.cpan.org a CNAME, or a mirror, of metacpan.org, also preventing having to have search.cpan.org be a machine just for emitting 301s.

    Would also be nice if the remaining missing modules/missing files/missing pod issues could be resolved before search.cpan.org is shuttered.

    ReplyDelete
  9. This is a shame. Personally I find search.cpan.org much easier to navigate and understand. Still, I guess if there's no struggle there's no progress.

    ReplyDelete
  10. search.perl.org is incredibly helpful to make Perl powerful. It is only recently that metacpan has shown up as a viable alternative. And while I have a slight preference for the presentation on search.perl.org when I want to read module docs (no space wasted for a left column), I understand that maintaining two overlapping tools isn't what you want to do for a long time, and also the metadata on metacpan often are very useful.

    But as others have said: I really hope that the name "search.cpan.org" will remain forever, even if the implementation will be done with metacpan code. It is one of the best trademarks Perl ever had.

    ReplyDelete
  11. End of an era, indeed.

    Still, I bristle at being left off of that as you have no idea the epic amount of time spent propping that site up for years at WU. Also, Ben Hockenhull hosted another system at Webster U. just down the street. Alan Burlison got Sun to donate the hardware, too.

    ReplyDelete
  12. Every time I search for "perl MODULE::NAME" search.cpan.org come up first. I would say 80% of the documentation I find goes to search*. Are these SEO seedings going to be broken after the move, or are there going to be redirect mappings to ensure the rankings stay relevant?

    ReplyDelete
  13. CPAN's web interface was an incredible feature and way ahead of its time. Other communities were still pointing people to FTP servers when CPAN had a full-featured search interface with fancy online documentation.

    I got a chance to work on the code back in the day - I added the grep and diff features and a few other things, I think.

    Good night, sweet prince!

    Sam Tregar

    ReplyDelete
  14. Nooooooo! :-O

    I likewise appreciate the work that search.cpan represents. As others, I have always preferred it to the metacpan site as a much cleaner interface.

    I'll have to change my Perl course too to reflect the change I guess. Hmmmm.

    ReplyDelete
  15. Thank you for many years of helpful package management resources! Saying 2005 era perl code is out of date makes me feel super old! I think that's the first year I started learning pearl. MUCH LOVE! AND RESPECT!

    ReplyDelete
  16. This headline may cause some readers (e.g. who see it via Hacker News) to believe that a pillar of Perl is being taken down as part of a perceived decline in Perl ... I wish it could be rewritten as "Perl Community Upgrades Interface to Modules; search.cpan.org Replaced by metacpan.org"

    ReplyDelete
  17. I have learned and benefitted so much from cpan.org over the years that I really cannot thank all of those who've contributed enough.

    It's going to be awhile before I learn to not type "cpan.org" into my address bar.

    ReplyDelete
    Replies
    1. cpan.org is not related to search.cpan.org. cpan.org is still the front page of CPAN and will continue to provide links to metacpan as it already has been.

      Delete
  18. Really appreciated using search.cpan.org over the years - I'm going to miss it!

    Thanks for everyone who contributed to coding it and keeping it going.

    ReplyDelete
  19. As a relative newcomer to perl, but having been a system admin for over 10 years, I've always felt search.cpan.org was a bit clunky, and I generally prefer to use metacpan because of its familiar Wikipedia-like interface.

    However, I appreciate the value search provides and I understand that the pros who have been using it for forever will find it hard to adjust.

    I am a bit shocked to find out that the code was never open sourced, given this is an open source community, and I am really surprised there's not more of a clamor from the aforementioned old pros to open source it now so that some dedicated fan could fork it.

    Given that it doesn't seem likely that will happen, could you at least work with the internet archive project to make a completecomplet rendering of the site before the transition, so we can still access it via the wayback machine? I know something similar was done for other sites (geocities anyone?)

    I'd like to be able to show my kids the site in 15 or 20 years when they are (hopefully) doing coding for a living themselves.

    ReplyDelete
  20. If Perl wasn't in decline before now, it certainly will be with the nightmarish interface of metacpan.org.

    When will the rest of cpan.org be taken down?

    ReplyDelete
  21. Really need to advertise this as 'New great updates coming to CPAN!' instead of the "eras are ending" stuff.

    ReplyDelete
  22. Hats off, many thanks for being a reference for at least two generations of Perl developers.

    ReplyDelete