An odd excerpt from the incoming spam pile:
Subject: transportation by air of day-old chicks to RussiaWe can offer available for charter flights, aircraft An-26B, for transportation by air of day-old chicks.
Our price from Europe to Russia from 10000 per one charter flight.
Best regards
Managing Director: Anatoly Ryabov
We finished the upgrades around 8pm (PST). It was surprisingly easy to upgrade from RHEL3 to RHEL5. The only thing we've run into so far were some of our custom built things we had compiled with the system OpenSSL that needed recompiling.
Most services are up again (and have been for some hours). We're upgrading our monitoring and system management tools and we're still tweaking and configuring, so do let us know if you see anything broken.
Friday, December 8th, 2007 will be our "annual" datacenter day. This means that Ask and I will go down to the colocation facility, swap drives, rewire things, upgrade BIOSes, upgrade operating systems, and reaffirm our frustration with hardware. (Our Cyclades Console Server means we hardly ever have to physically visit the machines.)
Starting at around 12pm PST (20:00 UTC) some perl.org services may be unavailable. Email to perl.org, cpan.org, and other hosted domains may be delayed. Some websites including PerlBug, CPAN Ratings, www.perl.org, www.parrotcode.org, www.perldoc.org, search.cpan.org, may be unavailable. We have a plan to minimize downtime for each service.
We expect the maintenance period to last less than six hours. Otherwise, we'll be ordering some pizza.
One of the biggest projects on our list for tomorrow is an upgrade to Redhat Enterprise Linux 5. Many of our boxes are currently running RHEL3, which is getting a little long in the tooth. We're looking forward to the newer kernel and more modern utilities.
We'll post here if there are any outstanding issues.
Our main internal fileserver has gone funky. Due to the joy of NFS, this is making other servers unhappy too. We suspect bad ram, but don't know yet. The machine is currently rebooting and fscking. We'll update this post as we learn more.
Update 8:54pm: The machine is back up, nothing appears to be horribly wrong, although we lost one disk from a RAID set. We're moving some data around, but things should be returning to normal shortly.
Update 10:16pm: Everything seems to be fine now. I expect those to be famous last words.
As our holiday gift to you, we provide you with some amusing monkey spam that made it through our spam filters at perl.org. This one was sent to a CPAN related email address, from a hosting provider's machine in Texas. The phone number has an Ohio area code, but is missing the last digit, so even if you were interested in a monkey, you'd need to write back to pastor patrick's Yahoo Mail account.
At perl.org we do our best to keep spam out of your mailboxes and off of our mailing lists. It's not easy, as the only thing worse than letting too much spam through -- is blocking legit email. Over the past month, we've made some changes to our mail system to ensure that mail continues to flow fast and furious... and we have some changes planned for the near future to continue to improve our systems.
And now... the monkey spam:
hello,
Complement of the season,
MY NAME IS joe helderberg, I need a responsible someone to adopt my baby capuchin , My baby is still available for christmas adoption and his 7weeks old now,weight 7lbs,Akc regis and am asking for $800.presently am in our church camp with my pets in Ohio.just tell me if you are still interest in my baby monkey so that i can tell pastor patrick about the baby shipment.
I will be waiting to hear from you if you are still interest asap.
God bless
more info.(614)xxx-xxx
I love when we get weird emails at perl.org. Here's one that came in recently...
Dirk
22.11.2007 15:51Kopie:
Thema: Enquiry PerlHello Joanna,
we got an inquiry from our customer about the following products:
1 Unit of Perl
Please send a formal quotation with specify terms and conditions for
delivery as well as availability, anticipated delivery time and delievery
charges/proccesing fees and dealer discounts.
We also need Information on which operating system the software is
running. Also we need informations on which language
do you´ll deliever the software. Please also let us know on which form
you´ll deliever the software: Boxproduct or electronicaly delievery.ATTENTION: We need your fax-number, because we don´t order online (
Internet security problems)!!Please send your quote to my attention at removed@removed.de.
For more informations about our company please have a look on our homepage
(http://www.removed.de) here you´ll find
an english company profil.Thank you in advance for quick response.
best regards
Dirk
Purchase department
Apparently nasty car crashes isn't the only news this weekend. We seem to have lost a drive in one of the machines serving perl.org. Of course, it's the drive that everything else depends on.
perl.org and cpan.org email and some websites may be non functional while we're repairing it. Check back here for more updates.
Update 6:40pm: Looks like things recovered with some remote percussive maintenance combined with patience. (Have we shared how much we love our Cyclades console servers and power management recently?)
We got a few mails asking about www.cpan.org being down. It's not actually hosted here (anymore). Pair.com is hosting it and they were informed about it earlier today (~5-6 hours ago I think). I'm sure they'll get it back as quickly as possible, but emailing us doesn't help. :-)
Our CPAN mirror is at cpan.perl.org.
For a long time we had a confusing thing where to get a svn.perl.org subversion (and groups.pm.org webdav) account you had to go to auth.perl.org and set it up there.
We've improved it a bit now so resetting your password (or just setting up the account initially) you just go to svn.perl.org. Whee!
Let me know (ask@perl.org) if you have any trouble with the new site.
Geez! This week it seems like everything is falling apart over here. Servers that have been running for hundreds of days are acting up. Configuration files last changed 3 years ago needs tweaking. Programs not changed for 4 years have bugs popping up.
We had an outage of cpan.org mail (mail bounced) early this morning and later in the afternoon many of the perl.org sites were unavailable for a few hours. One of our database servers got "stuck" and it had a cascading effect on some of the web services.
Apologies! IM me at xmpp:ask@plys.net if you see anything else amiss and we'll get it fixed in a hurry.
At the request of Michael G Schwern gravatar icons have been added to author pages on CPAN search.
Any author can get a gravatar icon on their page by registering at gravatar.com using their CPAN email alias.
The svn.perl.org server went kaboom. The kernel killed itself with lots of Out Of Memory errors.
I can't figure out the magic incantation to get into the Lights Out Management on that box (it's an old IBM server) and it appears we set it up to use that rather than the Cyclades console and power management.
Anyway - the datacenter staff has been sent to our cabinet to get it booted, so hopefully it won't be too long before it's back. Worst case it'll be up tomorrow morning (PST) when Robert or I go down there and find out what's going on (sorry, you will have to do without the svn server for 7 hours).
