The Blog of Harald

Critters’ Journey a Lesson in Currents”:http://www.washingtonpost.com/wp-dyn/articles/A5445-2003Aug30.html

bq. Back in 1992, a violent storm tossed 20 containers of rubber duckies off the back of a cargo ship halfway between China and Seattle, and they were quickly presumed lost at sea. Instead, it appears the castaways embarked on an epic 11-year swim across three oceans and half the globe.

A couple of researchers have taken the opportunity, and are using duck sightings to update their computer models. Cool!

[ Title and link shamelessly swiped from Brian Dickson. ]

Interesting log entries:

61.181.5.155 - - [31/Aug/2003:04:33:37 -0400] "GET /chk/archives/20030205152634.html"
61.181.5.155 - - [31/Aug/2003:04:35:23 -0400] "POST /chk/cgi/mt-comments.cgi"
61.181.5.155 - - [31/Aug/2003:04:45:06 -0400] "POST /chk/cgi/mt-comments.cgi"
61.181.5.155 - - [31/Aug/2003:04:45:10 -0400] "GET /chk/archives/000203.html"

It took almost two minutes for some loser to type the latest pr0n-based comments spam into my comments form and post it, then s/he spent 9 minutes previewing the comment before saving it? Yeesh.

I would have been much less surprised if it had all happened in seconds; Given the initial Google search for “blog 2003 august Name: Email Address: URL: Comments:“, it would be trivially easy to automate posting a comment via movabletype’s CGI interface. I suppose I should be happy that it isn’t automated; I’d have to then write the corresponding automated delete tool… *sigh.

It probably doesn’t matter, but <PLONK> regardless.

“Failure Is Always an Option”:http://www.nytimes.com/2003/08/29/opinion/29PETR.html?ex=1377489600&en=480627d339458d74&ei=5007&partner=USERLAND:

bq. But engineers do know that nothing is perfect, including themselves. As careful and extensive as their calculations might be, engineers know that they can err — and that things can behave differently out of the laboratory.

bq. If engineers are pessimists, managers are optimists about technology. Successful, albeit flawed missions indicated to them not a weak but a robust machine.

The constant struggle between engineering and management, in a nutshell. We see it in the software world, too, where marketing schedules and “first to market” pressures clash with our desire to create well-designed, well-coded, defect-free software.

It’s a healthy struggle. “Perfect is the enemy of good”…

I’ve knocked off 9K, or about 1/3rd, of the size of an average blog entry. I reduced my blogroll somewhat (mainly, I ditched all the A-list blogs that _everyone_ lists :-), I took out the wordcount stuff, and I dumped all the silly icons. It seems to be making a difference, although it’s a little early to tell.

I don’t know why I care, really; “roomie’s blog”:http://blog.org/ is pushing 50Mb/day with his 100Kb category archives :-) Still, a little efficiency here and there never hurts…

From “Edd Dumbill”:http://usefulinc.com/edd/blog’s “Things that Just Work”:http://usefulinc.com/edd/blog/2003/8/29#12:34 I was led to “Gallery Remote”:http://gallery.menalto.com/modules.php?op=modload&name=phpWiki&file=index&pagename=Gallery%20Remote. It’s a client-side interface to “Gallery”:http://gallery.menalto.com/ (the PHP-based photo album manager) that makes it a lot easier to upload a bunch of images to a Gallery site. The UI is a little rough around the edges (that’s standard for Java applications, alas), but it certainly does the job, and it’s much easier to drive than the web-based upload interface. Recommended.

The Commandments and immorality

bq. The first four of the commandments have little to do with either law or morality, and the first three suggest a terrific insecurity on the part of the person supposedly issuing them.

From Daring Fireball: Good Times

bq. Most corporations now have a CIO (chief information officer), whose clout is directly proportional to the number of people employed in the company’s IT division. More IT staff means a larger budget, and budgets are the rulers used to measure wangs in corporate America.

bq. It is thus in the interests of corporate IT staff to deploy technology that requires a large IT staff for maintenance.

The next section comparing plumbing to IT is hilarious…

One caveat; I know quite a few perfectly rational, capable IT people who are stuck with Outlook and Exchange because that’s what management wants, not because _they_ think it’s the best solution. It’s not always the IT people who get to make these decisions…

In the last week my blog traffic has gone from a sedate 4Mb/day to an average of 16Mb per day, with a peak last Thursday of 32Mb. A quick perusal of the logs shows that “this entry”:http://blog.cfrq.net/chk/archives/2003/08/17/blackout-satellite-images/ is extremely popular; it’s on the first page of Google for several obvious search terms. (My weblog is on the first page for several other interesting search terms too; I wonder what I’m doing to drive this?)

Anyway, while I was investigating, I looked at the per-page byte counts, using my nifty “MySQL-based Apache logs”:http://jeremy.zawodny.com/blog/archives/000407.html, and I discovered that an empty weblog posting here is around 26Kb in size! This is due to the sidebar (mainly the blogroll and the total word count), and then the HTML overhead. By the time you add in all of those little buttons, the byte count doubles.

Yikes! It’s a good thing that html compresses well, and that most browsers speak gzip; I have “mod_gzip”:http://webcompression.org/ installed, so the actual bytes_sent value is around 8K for most entries not including the buttons). I think it may be time to do some trimming.

There are some easy tricks that’ll get me 25% or so; remove the (superfluous) buttons; the humour is getting old anyway. Change or remove the total wordcount stuff, and trim the blogroll.

One gross hack would be to use frames, evil though they are; then most visitors would download the sidebar only when it changed, instead of for every entry. I don’t think that’ll really help much, since most of my traffic seems to be Google driven. Naughty googlers they are, too; the top traffic generators (after “blackout satellite pictures”) are still:

* “Female Nudity”:http://blog.cfrq.net/chk/archives/2003/01/06/female-nudity/
* “Trends in Playboy Models”:http://blog.cfrq.net/chk/archives/2002/12/20/trends-in-playboy-models/

Strangely, they’ve switched places from “the last time I looked”:http://blog.cfrq.net/chk/archives/2003/02/20/proving-once-again-that-its-all-porn-and-money/.

Anyway, I’m not too worried at this point. Even 16Mb/day isn’t much; we’re doing 8Mb/day of SMTP traffic in each direction…

The “Zombie Infection Simulation”:http://kevan.org/proce55ing/zombies/ would normally go in the sideblog, but it’s just so cute that I had to put it here instead. Go play with it.

(There’s a version where the humans fight back, which seems just as inevitable but leaves lots of dead bodies lying around :-)

In “How To Change Anything”:http://blogs.salon.com/0002007/2003/08/27.html, “Dave Pollard”:http://blogs.salon.com/0002007/ summarises “a paper by Dana Meadows”:http://www.wholeearthmag.com/ArticleBin/109.html. It describes how to change anything by using one of ten system leverage points. I like both his summary and the original paper; they resonate with my own personal systems-oriented view of the world.

Sadly, I find myself working mostly at the bottom of the list :-)

The rest of Mr. Pollard’s website looks interesting too; Another reason to crack open a new can of spare time?

“Greg’s Digital Retouching Portfolio”:http://homepage.mac.com/gapodaca/digital/digital.html has a bunch of interesting examples of modern image retouching. Each image is a rollover allowing you to easily see the before and after versions of an image.

Look at this site, and then _remember_ it the next time you’re looking at a magazine ad or billboard image.

The “bikini-clad model”:http://homepage.mac.com/gapodaca/digital/bikini/index.html is particularly telling…

We have a “new roommate”:http://blog.org/. Hopefully he won’t play the stereo too loud…

I have 19 entries that are still in “Draft” status, and an electricity rant brewing in the back of my head (and my pocket; I took notes). So over the next week or two I’m going to resurrect some of the old stuff. I just need a couple of material components…

bq. “Joi Ito’s Web: Email is officially broken”:http://joi.ito.com/archives/2003/08/14/email_is_officially_broken.html :

bq. I pronounce email officially broken. If 17 percent of legit email is being blocked by spam filters, it’s not officially working.

A conclusion drawn from a single study is bad enough, but a single study done _by_ email marketers? Surely there must be _some_ bias in the research, and therefore the conclusions. We already know that users’ definitions of “opt-in” are often very different from marketers’ versions (The fact that I give someone my e-mail address on a registration screen does _not_ mean I want them to declare every spammer under the sun a “carefully screened partner or affiliate”…)

Based on my logs, a rough estimate is that cfrq.net _refuses_ 1-2% of inbound email. We keep track of the source of blocked e-mail, and send a nightly report to users; The number of complaints I’ve received can still be counted on one hand. The rest is _all_ unsolicited bulk e-mail. The front line filters are very conservative, so I also run SpamBayes on my personal mailbox; it catches about another 10% (again, with a single-digit number of false positives since I installed it a year ago) . That too is all unsolicited (No really, I _like_ the current size of my penis, thank you).

bq. I don’t care what excuses people give. The people who made smtp should have thought more about host authentication and the people who made IPv4 should have made longer IP addresses. My guess is that there were people who were voicing concerns who had more vision.

Hey, the technology was invented in the dark ages when compared to today’s world. Nobody expected personal computers, never mind _cheap_ personal computers, never mind laptops and palmtops. SMTP even predates domain names, for goodness’ sake. SMTP was invented in an environment where the community was small enough that bad manners could be policed, never mind security. Now we have hundreds of millions of people on the ‘net; I think e-mail is holding up remarkably well, personally.

The problem here is not email or technology; it’s a bunch of people who think that SPAM is a viable business (and sadly, enough people click that it is). We solved the junk fax problem with legislation, and we’re still working on telemarketers. We’ll get around to e-mail eventually. In the meantime, e-mail marketers will just have to put up with the fact that some bad apples have spoiled the barrel for _them_. Normal, person-to-person e-mail is working just fine…

Some of my regular visitors recently complained that my site was down. Well, actually, it is up, but I’m having strange DNS issues.

I used to have three nameservers, in three separate physical locations; one of them is even on a separate ISP. However, two of my three nameservers are down. ns1.cfrq.net is now in my basement (switched off) due to the upcoming acquisition, and ns3.cfrq.net is down while the host site deals with floods etc. after the blackout.

This would normally be ok, since I still have one functioning nameserver. There was a bit of confusion between myself and the InterNIC over it’s _address_, but that has been cleared up. However, it seems that several places (notably both sympatico and rogers@home) can’t seem to contact the one remaining namserver (ns2.cfrq.net, 216.191.155.143), even though the glue records for the zone (in the GTLD servers) are correct, and the machine is up and running. I’d appreciate it if anyone could enlighten me (FWIW, I’m probably missing something obvious :-).

For now, I’ve reprogrammed the local firewall to a) answer to ns1.cfrq.net’s address, and b) redirect DNS queries to ns2.cfrq.net. This is the power of linux with iptables at work; bi-directional NAT is four lines of my firewall script :-). I won’t be able to do this forever, but with any luck ns3.cfrq.net will be in service again shortly, and I’ll figure out what’s wrong with the ISP nameservers.

In the meantime, I’m still looking for a home for ns1.cfrq.net, but that’s another story…

Eric Carroll pointed out this “NOAA News Online Story”:http://www.noaanews.noaa.gov/stories/s2015.htm where they have published NOAA and AFWA (Air Force Weather Agency) images from the DMSP (Defense Meteorological Satellite Program). The difference is pretty striking:

Before:
!http://www.noaanews.noaa.gov/stories/images/nightlights-081403-0121z3.jpg!

After:
!http://www.noaanews.noaa.gov/stories/images/nightlights-081503-0103z3.jpg!

Toronto, Ottawa, Buffalo and Rochester have all clearly vanished, while Montreal continues to shine brightly…

(Image Credit: NOAA/DMSP. More are available from “the NOAA page”:http://www.noaanews.noaa.gov/stories/s2015.htm ).

I took “a couple of photos”:http://www.cfrq.net/gallery/blackout-2003 before the batteries in my camera died on Thursday afternoon…

I’m still looking for a night-time satellite photo of the area from Thursday night; I think that would look really cool, especially when compared to the “normal nighttime view”:http://antwrp.gsfc.nasa.gov/apod/ap001127.html …

It has now been confirmed: the blackout was caused when a courier (or was it a security guard?) hit the EPO switch instead of the door unlock button…

<ducking>

A big “Thank you!” to my generous host, who not only cleanly shutdown my server during the blackout (it was on the UPS) but also brought it back up this morning before she went home for the day…

So at around 16:10 Thursday afternoon, the power went off. With a couple of quick cell-phone calls we determined that the power outage was city-wide; never a good sign, because there are three relatively separate grids in Toronto. It was only once were in the car, listening to the radio, that we found out that it was most of Ontario _and_ the north-eastern USA.

People were very well behaved. Traffic was pretty bad, but most drivers were stopping at all intersections, and many civilians were in the intersections directing traffic. It took us 1.5 hours to get home, about twice as long as usual. Lots of people ran out of gas, though; no electricity to run the pumps :-). Fortunately, we filled up on Monday.

We had a very nice BBQ dinner, and sat and played Pirateer (by candlelight) and talked until about midnight. Somewhere in there we dragged the kids outside to look up. It was a beautiful clear night, and (just before moonrise) we could actually see the Milky Way, from inside a large city! That doesn’t happen often. We saw mars, of course, and about 10 satellites and several shooting stars before the mosquitos drove us back inside.

The power at our house was still off this morning when we got up. Officials have declared a state of emergency, meaning only essential personnel are supposed to be going to work. There’s no GO Transit and no subways running. Of course, the financial companies consider everyone essential all the time (and the markets _are_ open :-). There are four of us in the office today, working on critical customer issues; everyone else is home.

The kids’ camp was up and running this morning, but they’ve been informed that there is a scheduled blackout at 1300; apparently they’re rotating power across sections of the city until they can get the full grid up and running. They’re asking everyone to turn off air conditioners and other non-essential things, but I wonder how many people will actually do so.

It was interesting walking through the underground. Tim Horton’s was open, and most food places were getting ready to open late. The mom ‘n’ pop businesses (print shops, shoe repairs, newsstands, etc.) were all open, but the large chain stores were all closed. That’s the difference between being an owner and an employee, I guess.

Anyway, we survived :-). We’ll probably lose the food in our fridge, but other than that, it’s been an inconvenient adventure, not a crisis. We were lucky; some people spent the night in elevators, and it took hours to evacuate the subway system…