svn 1.5.2

svn.perl.org is now running svn 1.5.2. 

Don't use Apache::Reload in production

Graham (who runs search.cpan.org) let us know that two of the machines appeared to be running very slowly, and he thought it might be NFS related, as everything else looked fine. 

Here's what we found looking at where requests to our NFS server were coming from...

oops.

Turns out Apache::Reload had been accidentally turned on on one of the production servers for geourl and *every* page hit was causing a stat of a few hundred .pm files.   Turned it off, and NFS load dropped by 80%, and everything went back to normal.

Tune in next week for "Don't use NFS in production".

(And yes, this entire post exists to show off the pretty Google Chart.)

Slow April Day

Twenty miles to the East, fires burn.

But here at perl.org headquarters, it's a slow day.

Not that we're not busy as usual, but today our primary webserver was running veeeeeeeeery veeeery slooooooooowww..

I think it's just one of those days.

Today's issues:

1. full partition makes database unhappy



2. effective DDOS attack by people misusing the ntp pool.  (No - you are NOT supposed to set your time via HTTP using our service.)



3. (when I thought it was all done) someone in Australia tried to scrape a million messages off of the cpan-testers archive.  All at once.  (Blocked!)

But everything's back to fast and normal now.....

Mailbag: Unified CPAN

Today's message came in to one of the CPAN support addresses.  We could tell him about Bundle::Everything (out of date as it may be), although I suspect he doesn't have a comfortable enough chair to read them all.

Hello dear Hack-o-holics !

Is this not possible ???
I want to download all available perl-modules in one bundle !?

It is crucial, to download every useful module separated.

One bundle and then I can read them in peace at home.

Thank you

Related module links on search.cpan.org

Documentation pages on search.cpan.org now contain a list of related modules. For an example see LWP::UserAgent

This list is determined by data created by Perlmonks.org about which modules are discussed together.

Domain names not for sale

We regularly get inquiries about the perl.org, perlmonks.org etc domains like the following:

We are interested in purchasing your domain name. If interested, please let me know what it takes to buy it.

If agreeable, we can finalize the deal through escrow.com Monday.

Our typical response:

Hi $spammer,

One billion USD. Non-sequential twenty dollar bills preferred, but not required.

You pay the escrow.com fees.

Thanks,

- ask

p.s. no, it's not really for sale.

goodbye cpan-testers, hello cpan-uploads

Today, at 4:11pm Pacific Standard Time, an era (or at least 9 years) ended.  The cpan-testers mailing list, recipient of a all CPAN tests submitted for CPAN testers processing, no longer resends all of those emails to its 50+ subscribers.  The last message to be distributed via email should have been #1049514.  Further messages will be kept in the archive, http://www.nntp.perl.org/group/perl.cpan.testers (also home to RSS and Atom feeds).  Eventually, we're going to transfer to a proper database backed system that doesn't use email at all.

Several people noted that they used cpan-testers to receive notification of new uploads to CPAN.  We've created the cpan-uploads mailing list to receive these notifications.     Archive/RSS/Atom is at http://www.nntp.perl.org/group/perl.cpan.uploads.  You can subscribe by sending a note to cpan-uploads-subscribe at perl.org.

No more email delivery of cpan-testers

Dear cpan-testers recipients,

  Effective Monday night, Feb 11th, 2008, after over a million test reports[1], we will no longer be supporting the delivery via email of cpan-testers test reports.  With an influx of over 3000 messages a  day, this was causing us to deliver almost 200,000 outbound messages, most of which nobody read.

cpan-testers test report submission is unaffected by this change.[2]

If you are currently subscribed to cpan-testers, you'll be unsubscribed, and your mail server will breathe a sign of relief.

  You can find test reports at:
    http://www.nntp.perl.org/group/perl.cpan.testers/
    http://www.nntp.perl.org/group/perl.cpan.testers/rss/posts.xml
    nntp://nntp.perl.org/perl.cpan.testers/
    http://testers.cpan.org/

Why are we doing this?  Things were chugging along smoothly, except every few weeks we'd have a problem where a recipient's mail server would stop accepting mail, and we'd very quickly end up with a large backlog of thousands of messages waiting to be delivered.  This would slow down delivery for all of our mailing lists.  Recently, this has been happening more and more often.  We analyzed the
problem, and came to the conclusion that the best solution was to discontinue outbound email.  This will let us focus more time on maintaining more important things.

If you have something that is dependent on receiving all of these emails, please let us know.

Thanks!

-R

Footnotes:
[1]  http://www.nntp.perl.org/group/perl.cpan.testers/2008/01/msg1000000.html

[2]  There are some volunteers working on a new system that will take email out of the picture altogether, creating a faster, more streamlined, and more consistent test database.

Email Glitch affecting pm.org and cpan.org

This past Thursday and Friday, we had an email glitch affecting cpan.org and pm.org email addresses.  The issue has been fixed, and mail is flowing normally once again.  If you sent a message to a cpan.org or pm.org address and it hasn't gone through, or if it bounced, you may wish to send it again.

Chicken Spam

An odd excerpt from the incoming spam pile:

Subject: transportation by air of day-old chicks to Russia

We can offer available for charter flights, aircraft An-26B, for transportation by air of day-old chicks.

Our price from Europe to Russia from 10000 per one charter flight.

Best regards

Managing Director: Anatoly Ryabov

Upgrades mostly done

We finished the upgrades around 8pm (PST). It was surprisingly easy to upgrade from RHEL3 to RHEL5. The only thing we've run into so far were some of our custom built things we had compiled with the system OpenSSL that needed recompiling.

Most services are up again (and have been for some hours). We're upgrading our monitoring and system management tools and we're still tweaking and configuring, so do let us know if you see anything broken.

"Annual" Maintenance - Dec 28th, 2007

Friday, December 8th, 2007 will be our "annual" datacenter day.  This means that Ask and I will go down to the colocation facility, swap drives, rewire things, upgrade BIOSes, upgrade operating systems, and reaffirm our frustration with hardware. (Our Cyclades Console Server means we hardly ever have to physically visit the machines.)

Starting at around 12pm PST (20:00 UTC) some perl.org services may be unavailable.  Email to perl.org, cpan.org, and other hosted domains may be delayed.  Some websites including PerlBug, CPAN Ratings, www.perl.org, www.parrotcode.org, www.perldoc.org, search.cpan.org, may be unavailable.  We have a plan to minimize downtime for each service.

We expect the maintenance period to last less than six hours.  Otherwise, we'll be ordering some pizza.   

One of the biggest projects on our list for tomorrow is an upgrade to Redhat Enterprise Linux 5.  Many of our boxes are currently running RHEL3, which is getting a little long in the tooth.  We're looking forward to the newer kernel and more modern utilities.

We'll post here if there are any outstanding issues. 

Smething broke (at least it's the day after a holiday)

Our main internal fileserver has gone funky.  Due to the joy of NFS, this is making other servers unhappy too.    We suspect bad ram, but don't know yet.  The machine is currently rebooting and fscking.  We'll update this post as we learn more.

Update 8:54pm: The machine is back up, nothing appears to be horribly wrong, although we lost one disk from a RAID set.  We're moving some data around, but things should be returning to normal shortly.

Update 10:16pm: Everything seems to be fine now.  I expect those to be famous last words.