At about 8:30 AM on Thursday October 30th, the hard drive failed. It was reporting itself on the bus, but with a garbage manufacturer string and a size of 0; not terribly useful.

Fortunately, I had a backup; I use rsync to keep a copy of the server filesystems on a machine at home. I even had to restart the backup by hand on Wednesday morning, so we only lost about 24 hours worth of stuff (the Thursday morning backup was still in its early stages when the disk crashed).

Unfortunately, I lost all of my MySQL databases. Most of them are replaceable, but it’s still annoying. It seems that mysqlhotcopy “knows” that a list of tables fetched from the database is not quoted, so it does quoting itself. Unfortunately, a newer version of MySQL (or DBD::mysql) appears to be quoting table names. Net result:

DBD::mysql::db do failed: You have an error in your SQL syntax near ‘` READ, `apachelogs`.``access_blog_org`` READ, `apachelogs`.``access_fywss_com``’ at line 1 at /usr/bin/mysqlhotcopy line 438

So, no MySQL backups… *sigh.

Anyway, 40Gb drives were on sale at ICCT, so I ran out last night and picked one up, restored the backup, and replaced it in the server this morning. The old drive is about 15 months old and is still under warranty; Maxtor is sending me a replacement even as we speak, so I’ll end up with a spare drive at the end of this whole mess.

Moral: keep good backups :-)

Update: There’s some evidence that the last complete backup was actually on the 26th of October. I’ll have to investigate, because I know it’s been running every day since then…

posted at 2:07 pm on Friday, October 31, 2003 in Site News | Comments Off on Crash!

No Comments

No comments yet.

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.