SimPlanet2

Today (and yesterday), I spent some time trying to upgrade the Planet! software that runs Planet Perl and friends to the latest version.



I said "trying", because I'm declaring the upgrade a failure.

It appears to be because I'm testing by running two "update"s in a row, and it doesn't like rapid updates within the same minute or something, but it seems to randomly decide to only include certain people in the output. (And no, it's not always the same people.) That just isn't good enough. I could dive into the python code and fix it, but there are many other things on my todo list, and the old code works just fine, 99% of the time. It was also complaining bitterly about the RSS2.0 <items> block, so I had to tweak it for that.

Of course, instead of giving up, I got persistent, and it seems to be working ok now.

So... lets start over....

Tonight, I upgraded Planet Perl (and Parrot, and Six Perl) to the latest, greatest, Planet! Feed Aggregator.

While I was at it, I took the time to tweak the look of Planet Perl. (Thanks for the comments on the previous post.)I fixed up the CSS a bit, although it still needs more work. We're almost at the point where we can have alternate stylesheets. I did make one controversial change, which is to have it be full-width. The fixed-width we were using was a little too narrow, and picking another arbitrary width seemed silly.

Anyway, now I can leave this alone for a while.

SimPlanet

I don't like how Planet Perl looks. There's too much content, its too vertical. And it doesn't help that PlanetPlanet is showing stories from last June.

Over the past few days I've played with trying to only show short summaries, alternate sidebars and fancy javascript controls. But I didn't find anything I liked.

In a few days I'll come back to this. Anyone got any new ideas?

Planet Perl June Update

Welcome perlcast and Chip Salzenberg to Planet Perl.

I'll admit to not having listened to any of Josh McAdam's perlcasts yet, but they look cool. Chip has been writing up some very interesting things about Parrot recently.

More mail than you can shake a stick at

Today perl.org absorbed almost 30,000 spam and virus emails.

About 1/3 of that is viruses caught by clamav.

Plus at least another 12,000 invalid messages we never accepted. And I'm sure there's another huge bunch thats not getting logged properly

Then there's the huge portion of the internet that's outright blacklisted.

Without qpsmtpd, we'd be lost.

oops

I rebooted a bunch of our servers earlier today to pick up new kernels. We have our own build of Perl, Apache and other software like that mounted via NFS and on a couple of the boxes the NFS directory didn't get mounted when it booted.

Also on one of the cpansearch boxes squid didn't start up on boot, and it wasn't in our monitoring setup so I only noticed just now because a user wrote in! Doh. (It worked half the time, depending on which server you got).

Our colobus server also didn't start up properly (or rather logging didn't), so the nntp.perl.org server has been flaky through the afternoon.

Anyway, it should all be better now. If not, send us mail.

pgeodns

There's a barebones website for pgeodns now.

Robert moved our fancy DNS server out to a public repository, so it's availble from subversion if you are interested.

eww-tee-fff eight

Today some complaints came in about all the Perl Monger mailing lists switching to UTF-8. (As part of a fix for the encoding issues our Chinese group was having.)

So... I switched everything back to us-ascii, and created a special English (UTF-8) variant to use where necessary.

EU search.cpan.org mirror live

I flipped the switch, so cpansearch.perl.org/search.cpan.org is now using the EU mirror for European users.

(see the search.cpan.org mirror post from yesterday)

Thanks again to Digital Craftsmen. We're also running one of our "geodns" thingies on their server now.

Subversive Ponies!

Ponie has moved from CVS to Subversion. You can now find it here. This leaves only a few small projects running off of our CVS server.

search.cpan.org mirror

Over the last week Graham and I have been setting up the box provided by Digital Craftsmen; our first mirror not hosted on the perl.org infrastructure. We are planning a mirror in Japan and another US mirror too (but haven't gotten the hardware for it yet).

It's not on the search.cpan.org name yet, but (Very Alpha-ish) at cpansearch-geo.perl.org. If you are in Europe you should reach the new mirror, if you are anywhere else you should reach one of the perl.org hosted boxes (with the "hardware by bizrate" logo). The cpansearch-geo.perl.org name will go away when we are done testing.

Please try it out and let me know if you are having trouble with it (email ask@perl.org).

Our geography-aware DNS server, pgeodns, is a bit naively assuming that all of Europe will have better access to the UK mirror than to the West Coast US mirror.

In the process of setting it up, I updated pgeodns a bit, so if I made a mistake or two it might impact the ftp.perl.org and ftp.cpan.org names too, so please let me know if you are seeing any abnormalities with the DNS for those too.

pgeodns is in a private svn repository, but Robert says he'll move it to another one for me any day now. :-)

More disk!

More information and details later, but I wanted to share our new terabyte RAID with you:

[root@x4 root]# df -h /mnt/raid
Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              1.4T   33M  1.3T   1% /mnt/raid

Data center day / new hardware

Tuesday afternoon is going to be datacenter day™ for Robert and I.

Our colocation facility asked us to get off the building UPS as the building apparently is running out of UPS power.  Since they are incredibly nice to us (they just gave us the neighbor rack so we have two racks there now!) we could hardly say no.  Last week we figured out what we need (I hope!) and ordered a APC SUA3000RM2U UPS that should have enough capacity.  The building has a generator that'll kick in very quickly so we only need a minimal run-time.  If it doesn't have enough capacity we'll get another one of those most likely.   The higher capacity ones gets a lot more expensive per watt as it quicky turns into some 208V+transformer mess.

Monday morning I heard the doorbell/barking dog duo and Vani came upstairs telling me there was a DHL guy with a heavy package that he was going to try getting down to the house.  I rushed and caught him going down the stairs so he could help me bring it up again and put it in the car.  The darn thing is 120lb!

Later in the day UPS showed up with a Dell 5324 gigabit switch (I mentioned that before) and Fedex showed up with more drives for the file/NFS/database/backup server (more on that later).

Tomorrow I have to go by the post office and pick up a 48-port 100Mbit switch I found on eBay and then we'll be off to the datacenter to setup the new rack.

Many or most of our services may be down briefly Tuesday afternoon (local time in California) as we shut down equipment to move it over to the UPS and install the new disks.

Challenge Response for NNTP

A few weeks ago we enabled posting to perl.org mailing lists via Google Groups. In looking to expand that to the rest of the NNTP enabled world, we've settled on implementing a Challenge/Response system for verifying email addresses. (It really isn't fun when you reply on a mailing list and find out the email address you're replying to is fake.) Because we generally require posting from the subscribed address to the perl.org mailing lists, we know those are valid addresses.

So... Last night I attempted to implement TMDA. It is very easy to setup... but... I hit a roadblock. The envelope from/return path on NNTP gated postings is generally the gateway, not the original poster. So, TMDA had to be modified to use the From address instead. I had it mostly working after a few hours, alternately tweaking the scripts (something I didn't want to do -- makes it hard to upgrade) and writing procmail wrappers. Finally, I gave up, after TMDA made it clear that it wasn't happy if I sent a message from address A, and then sent the verification message from address B.

Next step is either to:

a) write my own Challenge/Response system just for this.

b) find another system that works

c) give up, letting the 3 or 4 people a wek who post via a NNTP server that is not nntp.perl.org send their emails directly to the mailing list.