The Blog of Harald

I decided to try out “Linode”:http://www.linode.com (300Mb RAM, 8Gb disk space for $20/month), since they support Ubuntu 7.04 and have a better memory model and kernel support. So far so good; persephone.cfrq.net has been a virtual server since Wednesday! The migration was trivial; I bought a new linode installation, booted it up, and used rsync to copy files from the old server to the new one. It took about 6 hours altogether. Then I shutdown all the services on the old box, did one final rsync, and updated the DNS to point to the new server. Everything came up right away; I’m not sure if anyone even noticed the change :).

I haven’t decided if I’m going to keep it here or not yet, but I’ve bought myself some procrastination time!

It’s been 5 long and blissfully problem-free years, but now it’s time to find a new host for persephone.cfrq.net. For various unimportant reasons, my server needs to move out of the server room it’s been hiding in. I have a few options:

* see if I can find another free host. Unlikely.
* follow Reid’s footsteps and switch home ISPs to one that allows servers. He’s had a lot of problems with that over the years, but I’ll consider it.
* Move to a virtual server provider (e.g. Linode or Tektonic). I already have a Tektonic VPS, but now we’re talking about real money ($15-$20 per month).

I may have to evict some of my high-resource tenants in the process, which sucks… Still, I can hardly complain, having found free hosting in one place or another for about 15 years now!!!

I just wrote a simple wordpress filter that automatically creates links to a post if the title of that post (or other ‘link names’ in the post’s metadata) appear in text. I wrote it for the “Queen’s Guard”:http://www.the-gang.ca/roleplay/queensguard/ game log. It gives us a little bit of wiki-ness without having to use a full-blown wiki; I haven’t found any wiki software that I like enough to foist on my non-technical gaming buddies.

If anyone’s interested I can clean it up and post it somewhere.

(Update: It’s published: “Title Links Plugin”:http://blog.cfrq.net/chk/archives/2006/08/02/title-links-plugin/)

It’s amazing how quickly people stop stealing your bandwidth when you substitute the images they were “borrowing” with something relatively disgusting.

I’m evil, it’s true. On the other hand, the traffic load from the image theft was bringing down the server, so…

“The Queen’s Guard”:http://www.cfrq.net/~rolemaster/queensguard/

Not much there yet, but knowing Rob, there will be real soon now…

The patches originally scheduled for June are finally done! Causes were many; a steady stream of new defects, staff defections, and general overload (both my team and QA :-). Now we just have to get all of our other “before the end of the year” commitments done before the end of the year.

That’s part of the lack of activity around here; another part is persephone 2.0, currently about half-constructed in a VMware partition on the home machine. I’m upgrading all the way from RedHat 7.3 to Ubuntu 5.10. That’s not really an “upgrade”; instead I’m building a new disk image from scratch, and then carefully migrating the configuration and data for each user and service. I’m hoping to be done by the end of the year…

I’m sure it’d be easier to just buy a new machine and put them up side-by-side. I’m not sure how Michelle would feel about that, though :-).

The upgrade to WordPress 1.5 was painless. I had to:

* backup :-)
* install the new version
* run the database upgrade script
* get a new version of the textile 2 plugin
* copy my old index.php etc. files into a theme directory, edit them, and activate the theme

That last step took most of the time, but it now seems that the old blog and the new blog look the same, so…

tah-dah!

I finally figured out “where all the spam is coming from”:http://www.google.com/blogsearch.

The coincidence is too perfect to ignore.

Ain’t progress grand?

I desperately hope that there is a special section of hell reserved for comment spammers…

The construction crew managed to fill the server room with drywall dust, taking out four power supplies (including persephone.cfrq.net). It took a couple of days to procure and install a replacement.

As always, I thank my generous hosts for their time and energy!

At the rate that i’m blacklisting spammers, I’m going to end up blocking the entire Internet… *sigh.

GO AWAY! My statistics pages (the only place you’ll see referers) are password protected! Google can’t see them! You won’t get any pagerank from me!

(pant pant pant…)

Today I’m feeling like throwing in the towel on this web server business: there’s just too much crap to deal with.

A friend’s server was broken into and defaced last week by a script kiddy. I’ve been double-checking my box over the last few days, and I’m astonished at the amount of crap flowing in from the Internet. As a security professional I knew it was bad, but I was fooling myself; I didn’t know it was _this_ bad!!!

I monitor the site regularly, mainly to ensure that we’re not abusing bandwidth that is generously donated, but also to make sure everything is working, and to watch for obviously suspicious activity. In the last week a major portion of the traffic to this server has been:

* referrer spam (which doesn’t do anything for the spammer, since I don’t display referrers anywhere; it only abuses my bandwidth). About 15% of my bandwidth for the last _month_ has been referrer spam; they seem to breed faster than I can block them out!
* people trying comment spam on weblogs with no comments (and no comment script!). This includes attempts to invoke old security holes in Movable Type.
* people probing for security defects in software that I don’t even have installed.
* people probing for security defects in software that I _do_ have installed (fortunately that was password protected, so they didn’t get in :).
* probes for network sockets (both for software with vulnerabilities, and for software installed by hackers). This box is heavily firewalled (in both directions; blocking outbound traffic has saved my bacon more than once!), but I still see the logs.
* password guessing attempts (mainly via SSH, which has been locked down to a small number of IP addresses for months now, since the last major SSH vulnerability).

The promise of Open Source software was that more eyes staring at code would lead to fewer defects. I’m seeing the opposite; it seems that the rate of vulnerability annoucements, and resulting patches, is _increasing_. Just last week I just upgraded three packages here as a direct result of security announcements (and, as mentioned above, caught someone probing for one of them…)

The Internet has become the cesspool predicted in several recent science fiction novels (notably Peter Watt’s Behemoth, which specifically mentions automated virus / hacking activity). After three days of looking two closely at my logs I feel like pulling the plug. If it were just me using the server, I probably would…

I’m testing new processing for conditional gets for RSS feeds.

Pay no attention to the man behind the curtain…

are re-enabled. I forgot to leave a note to myself to turn them back on; Irving had to remind me :)

It’s Christmas day, the spammers are out in force, and I don’t feel like babysitting. Comments are kaput, for now.

For about two weeks now I’ve been getting massive amounts of comment spam, both here and on some old Movable Type weblogs on the site. None of the comments appear to be making it through; WordPress has comment moderation (which just makes it annoying to delete them), and the MT sites have comments turned off. (For some reason the comments are still getting posted; but they’re not displayed on the blog pages, so no major damage).

Most of the comments are “Hey, what a great website!” comments, with no URLs (and no racy referrals, either); I don’t understand the point. Are they just trying to be annoying? Is this a test? Did the comment spammers suddenly find new tactics about two weeks ago, or did they suddenly notice my website again?

I guess it’s time to research WordPress blacklisting tools…

If you’ve seen layout problems or certificate warnings recently, it’s because WordPress has two URLs for your weblog, that are _supposed_ to be for different purposes. Unfortunately, the code appears to use the two interchangably.

When I changed what was supposed to be the admin interface from HTTP to HTTPS (so that the naive referer check in wp-admin/admin-functions.php would work with my SSL-based webserver), it apparently broke _some_ (but not all) of the URLs on the main blog page.

I don’t have time to investigate right now, but I’ll get around to it eventually.

It took _way_ too much effort to get those blobs of code to display properly. Even inside PRE and CODE tags, WordPress was mangling stuff. I downloaded “David House’s grabcode plugin”:http://xmouse.ithium.net/archives/2004/07/19/implementing-a-code-snippet-system, but then I had to play with my CSS to get the results to look acceptable (and they’re still wrong in the RSS feeds, but I give up for now).

It’s done now, and the _next_ time will be easy :-)

I was looking at my logs, because I was seeing a lot of traffic on my XML feeds. Clichéd, perhaps, after the Microsoft debacle :-). I was looking because my RSS2 feed had risen to the top of the daily traffic statistics.

I did notice that many aggregators still aren’t using Conditional GET, and that some others don’t support gzip. But that wasn’t it… Instead, a full 25% of hits resulted in HTTP Code 302 (not modified), and yet were _also_ transferring the full RSS feed (sometimes compressed, sometimes not). Digging in the code, I found (in the Conditional GET logic in wp-blog-header.php):

I moved the exit to the right place, and now all my Conditional GET clients are downloading 0 bytes :-). Here’s the fixed version:

I wonder who is teaching people that it is acceptable to download the entire content of a website, as fast as possible, and then come back and do it again the next day (and the next)…

larbin has been added to the list of blocked robots. Buh-bye!