Wednesday, December 26, 2007

Smething broke (at least it's the day after a holiday)

Our main internal fileserver has gone funky.  Due to the joy of NFS, this is making other servers unhappy too.    We suspect bad ram, but don't know yet.  The machine is currently rebooting and fscking.  We'll update this post as we learn more.

Update 8:54pm: The machine is back up, nothing appears to be horribly wrong, although we lost one disk from a RAID set.  We're moving some data around, but things should be returning to normal shortly.

Update 10:16pm: Everything seems to be fine now.  I expect those to be famous last words.

No comments:

Post a Comment