Thursday, December 25, 2008

Merry christmas; down!

Whoops.  We had one outage earlier that dragged down a server (OOM killer went nuts).  I rebooted it and of course it won't come back up.  Either the (raid'ed) disk got corrupted, the bios ran out of battery and lost some critical settings or some variation of that.  I can't get into the BIOS to make it netboot remotely.

In any case, I'm in Europe and Robert might have the day off (imagine!), so all the services (including and running on that box might be down for the day.

Enjoy the holidays.  :-)

Update - Robert got it fixed within a couple of hours.  Yay.  (ps. SELinux is really annoying).

Friday, May 9, 2008

Don't use Apache::Reload in production

Graham (who runs let us know that two of the machines appeared to be running very slowly, and he thought it might be NFS related, as everything else looked fine. 

Here's what we found looking at where requests to our NFS server were coming from...


Turns out Apache::Reload had been accidentally turned on on one of the production servers for geourl and *every* page hit was causing a stat of a few hundred .pm files.   Turned it off, and NFS load dropped by 80%, and everything went back to normal.

Tune in next week for "Don't use NFS in production".

(And yes, this entire post exists to show off the pretty Google Chart.)

Monday, April 28, 2008

Slow April Day

Twenty miles to the East, fires burn.

But here at headquarters, it's a slow day.

Not that we're not busy as usual, but today our primary webserver was running veeeeeeeeery veeeery slooooooooowww..

I think it's just one of those days.

Today's issues:

  1. full partition makes database unhappy

  2. effective DDOS attack by people misusing the ntp pool.  (No - you are NOT supposed to set your time via HTTP using our service.)

  3. (when I thought it was all done) someone in Australia tried to scrape a million messages off of the cpan-testers archive.  All at once.  (Blocked!)

But everything's back to fast and normal now.....

Thursday, April 17, 2008

Mailbag: Unified CPAN

Today's message came in to one of the CPAN support addresses.  We could tell him about Bundle::Everything (out of date as it may be), although I suspect he doesn't have a comfortable enough chair to read them all.

Hello dear Hack-o-holics !

Is this not possible ???
I want to download all available perl-modules in one bundle !?

It is crucial, to download every useful module separated.

One bundle and then I can read them in peace at home.

Thank you

Saturday, April 5, 2008

Related module links on

Documentation pages on now contain a list of related modules. For an example see LWP::UserAgent

This list is determined by data created by about which modules are discussed together.

Friday, February 22, 2008

Domain names not for sale

We regularly get inquiries about the, etc domains like the following:

We are interested in purchasing your domain name. If interested, please let me know what it takes to buy it.

If agreeable, we can finalize the deal through Monday.

Our typical response:

Hi $spammer,

One billion USD. Non-sequential twenty dollar bills preferred, but not required.

You pay the fees.


- ask

p.s. no, it's not really for sale.

Monday, February 18, 2008

goodbye cpan-testers, hello cpan-uploads

Today, at 4:11pm Pacific Standard Time, an era (or at least 9 years) ended.  The cpan-testers mailing list, recipient of a all CPAN tests submitted for CPAN testers processing, no longer resends all of those emails to its 50+ subscribers.  The last message to be distributed via email should have been #1049514.  Further messages will be kept in the archive, (also home to RSS and Atom feeds).  Eventually, we're going to transfer to a proper database backed system that doesn't use email at all.

Several people noted that they used cpan-testers to receive notification of new uploads to CPAN.  We've created the cpan-uploads mailing list to receive these notifications.     Archive/RSS/Atom is at  You can subscribe by sending a note to cpan-uploads-subscribe at

Sunday, February 10, 2008

No more email delivery of cpan-testers

Dear cpan-testers recipients,

  Effective Monday night, Feb 11th, 2008, after over a million test reports[1], we will no longer be supporting the delivery via email of cpan-testers test reports.  With an influx of over 3000 messages a  day, this was causing us to deliver almost 200,000 outbound messages, most of which nobody read.

cpan-testers test report submission is unaffected by this change.[2]

If you are currently subscribed to cpan-testers, you'll be unsubscribed, and your mail server will breathe a sign of relief.

  You can find test reports at:

Why are we doing this?  Things were chugging along smoothly, except every few weeks we'd have a problem where a recipient's mail server would stop accepting mail, and we'd very quickly end up with a large backlog of thousands of messages waiting to be delivered.  This would slow down delivery for all of our mailing lists.  Recently, this has been happening more and more often.  We analyzed the
problem, and came to the conclusion that the best solution was to discontinue outbound email.  This will let us focus more time on maintaining more important things.

If you have something that is dependent on receiving all of these emails, please let us know.




[2]  There are some volunteers working on a new system that will take email out of the picture altogether, creating a faster, more streamlined, and more consistent test database.

Saturday, February 9, 2008

Email Glitch affecting and

This past Thursday and Friday, we had an email glitch affecting and email addresses.  The issue has been fixed, and mail is flowing normally once again.  If you sent a message to a or address and it hasn't gone through, or if it bounced, you may wish to send it again.

Sunday, January 20, 2008

Chicken Spam

An odd excerpt from the incoming spam pile:

Subject: transportation by air of day-old chicks to Russia

We can offer available for charter flights, aircraft An-26B, for transportation by air of day-old chicks.

Our price from Europe to Russia from 10000 per one charter flight.

Best regards

Managing Director: Anatoly Ryabov