So another comment spammer took down my server this weekend. It seems that it takes Movable Type over 8 seconds to rebuild the blog.org pages after a comment has been posted (the category pages are large, and get re-written to update the comment count in the summary). If a spammer tries POSTing to several comments pages at the same time, or does so over a relatively short period of time, I get a whole bunch of mt-comments.cgi scripts running simultaneously.
At 8Mb (of working memory) each, it doesn’t take long for them to max out the memory on my wimpy 128Mb box, at which point paging starts, slowing everything down and making the problem worse. As more HTTP requests show up, and cron scripts run, the box starts thrashing (i.e. spending all of its resources moving pages in and out instead of accomplishing useful work). I couldn’t even SSH into the box; the SSH negotiation was timing out after a few minutes.
Usually I have to ask my host to physically reset the server, but this time it was a long weekend. Fortunately I had a remote shell lying around. But it took two _days_ to run su, type my password, and kill off the offending httpd and mt-comments.cgi processes. In the meantime, many other important daemons had been killed due to out-of-memory, and the box was completely ignoring web requests and e-mail sessions; in short, the machine was a mess.
MT-Blacklist is due out today, and I intend to install it, but it won’t help this problem; by the time mt-comments.cgi is being exec()ed, it’s already too late.
So instead I wrote a simple locking wrapper for mt-comments.cgi. It’s in C, so it’s tiny (working memory is 306Kb instead of 8Mb; still way too large, but much better). It grabs a lock file before running mt-comments.cgi, so that only one instance is running at a given time. I’m hoping this will prevent the box from falling off the ‘net the _next_ time a comment spammer shows up.
I’ve also dropped the value of MaxClients in my Apache config, to prevent too many simultanous Apache processes from starting up (since this will also eat the virtual memory system for breakfast).
I wish there was a better way to do load shedding in this context, but I can’t think of one off-hand…
Sorry it’s boring you – what would make it exciting? More comments? ;-)
For twenty-five points, what 80s pop lyricist wrote:
“Life was easier when it was boring.”
As for “bored now”, yeah, I think blogs need to scratch an itch. Raymond Chen keeps writing “The Old New Thing” to explore how we got here (where “here” is the current tangle of Microsoft technologies). Jon udell blogs because he gets paid to write about the bleeding edge, and the only way to do that is to play there. Miles Thibault (student of mine at U of T) has just started a blog (on my orders) where he’ll write about his explorations of C-Python’s implementation, and so on. So, what’s your itch?
(The Following Is A Completely Unedited Response Dictated Through Viavoice; Lack Of Editing Was Prompted By The Unfortunate Discovery That “Press Delete” Is Sometimes Misinterpreted As “Press Escape”, Which In Livejournal And Mt Comments Has Disastrous Consequences. Apologies In Advance For Incomprehensibility. )
Now that was interesting; why would ViaVoice capitalize that macro?
Anyway, I’ve been blocking for about seven years knell endive fine to the my interest comes and goes. I have found that adding photos makes it more interesting for me, as well as giving myself an assignment topic.
For me, however, one of the major incentives (before my tendinitis made it hard to write as quickly as I think because I have to use ViaVoice) Was to improve my writing and my ability to write even when I didn’t feel like writing. the latter is an extremely useful skill for someone who writes for a living, especially magazine writing.
what are your favorite kinds of entries, the ones you most in choy writing? Perhaps you could focus on those kinds of entries.
Wow, I have no idea how much of the above is going to be understandable. :-)
Apologies for this completely contentless comment whose sole purpose is to enable me to enter my correct personal posting information for future use.
and of course I entered it incorrectly. Here’s another attempt.
While were on the topic of movable type comments, have you had any problems with spam postings in your comment sections (not counting mine, of course :-)). I’m starting to encounter them more, comments masquerading as actual remarks about my blog, but in truth are just links to commercial sites.