Graham (who runs search.cpan.org) let us know that two of the machines appeared to be running very slowly, and he thought it might be NFS related, as everything else looked fine.
Here's what we found looking at where requests to our NFS server were coming from...
oops.
Turns out Apache::Reload had been accidentally turned on on one of the production servers for geourl and *every* page hit was causing a stat of a few hundred .pm files. Turned it off, and NFS load dropped by 80%, and everything went back to normal.
Tune in next week for "Don't use NFS in production".
(And yes, this entire post exists to show off the pretty Google Chart.)
