The Blog of Harald

I’ve been a big fan of computer automation for as long as I’ve been computing. Computers are partcularly well suited for repetitive, mundane tasks. I used to have a system set up at Alias, based on “track”:ftp://ftp.cs.toronto.edu/pub/track.README, that would automatically keep all of the system software and configuration files up-to-date, long before HP Software bought Radia :-). My PVR is probably my current ultimate example; it automatically downloads TV listings from the source, searches those listings for my favourite shows, resolves conflicts, etc.; all without my input!

Anyway, it always surprises me when I power on an infrastructure server, and the LDAP server or Perforce server or whatever isn’t configured to start automatically! I mean really; who runs around manually starting essential services after a power outage? I thought we stopped doing that in the 1970s…

(We had to power down our labs over the weekend, because the A/C cooling tower on the roof was being refurbished. It took me all day today to get everything running properly again…)

“UNIX – The Hole Hawg”:http://www.team.net/mjb/hawg.html – I’ve been a UNIX person since I started university, what, 22 years ago now?

I was asked for my title links plugin (See “wordpress filter”:http://blog.cfrq.net/chk/archives/2006/06/10/wordpress-filter/), so here it is:

“title-links-1.0.zip”:http://blog.cfrq.net/chk/wp-content/uploads/2006/08/title-links-1-0.zip

Updated with a brute-force fix for a conflict with the “pretty quotes” functionality in Textile:

“title-links-1.1.zip”:http://blog.cfrq.net/chk/wp-content/uploads/2008/05/title-links-1-1.zip

This plugin will create links between WordPress posts whenever the title of the target post appears in the text of another.

You can specify alternate link keywords for a post by creating a Custom Field named ‘link_names’, containing a set of alternate phrases surrounded by square brackets. e.g. if the ‘link_names’ field for the post titled “Lady Danielle de Barbarac” contains

[Lady Danielle] [Danielle]

Than any occurence of the either phrase in another wordpress post will create a link to the Lady Danielle article.

You can see this plugin in action on “The Queen’s Guard”:http://www.the-gang.ca/roleplay/queensguard/ and “Five Rivers”:http://www.the-gang.ca/roleplay/fiverivers/ game log sites.

I just wrote a simple wordpress filter that automatically creates links to a post if the title of that post (or other ‘link names’ in the post’s metadata) appear in text. I wrote it for the “Queen’s Guard”:http://www.the-gang.ca/roleplay/queensguard/ game log. It gives us a little bit of wiki-ness without having to use a full-blown wiki; I haven’t found any wiki software that I like enough to foist on my non-technical gaming buddies.

If anyone’s interested I can clean it up and post it somewhere.

(Update: It’s published: “Title Links Plugin”:http://blog.cfrq.net/chk/archives/2006/08/02/title-links-plugin/)

The sudden absence last week was due to a J2EE course; come the winter I’ll be adding a J2EE-based product to my CPE stable.

Our instructor was very good; he didn’t waste time with piddly details that are in the API docs anyway (except the important ones); instead he explained what the pieces are *and* how they all fit together! Suddenly, it all makes sense! Actually, used carefully, J2EE is pretty cool…

And he made me snort coffee when he mentioned the two types of session bean: _stateless_ and _useless_ :-)

bq. Maintaining badly written code is like trying to solve a crossword puzzle set by someone who can’t spell.

(taken from “Simon”:http://simon.incutio.com/archive/2005/03/18/quotes)

I’ve been trying to merge a series of changes from the mainline to the bugs branch all week. But every time I get into the flow, some critical interrupts would arrive. Even working from home didn’t help; the phone rang off the hook all day. (I’ve got two children in school nearby, so I can’t just turn off the ringers, alas). I’ve had to drop and restart the task so many times that it got completely befuddled, to the point where I realized this afternoon that I had to start over. On the plus side, I know what I need to do now, so it won’t take me another entire week… unless more interrupts show up!

I’m glad I’ve got two other team members now; at least _they_ are working on long-term projects, and are able to concentrate on one thing at a time. Without that, I don’t think we’d be making any progress at all…

It took _way_ too much effort to get those blobs of code to display properly. Even inside PRE and CODE tags, WordPress was mangling stuff. I downloaded “David House’s grabcode plugin”:http://xmouse.ithium.net/archives/2004/07/19/implementing-a-code-snippet-system, but then I had to play with my CSS to get the results to look acceptable (and they’re still wrong in the RSS feeds, but I give up for now).

It’s done now, and the _next_ time will be easy :-)

I was looking at my logs, because I was seeing a lot of traffic on my XML feeds. Clichéd, perhaps, after the Microsoft debacle :-). I was looking because my RSS2 feed had risen to the top of the daily traffic statistics.

I did notice that many aggregators still aren’t using Conditional GET, and that some others don’t support gzip. But that wasn’t it… Instead, a full 25% of hits resulted in HTTP Code 302 (not modified), and yet were _also_ transferring the full RSS feed (sometimes compressed, sometimes not). Digging in the code, I found (in the Conditional GET logic in wp-blog-header.php):

I moved the exit to the right place, and now all my Conditional GET clients are downloading 0 bytes :-). Here’s the fixed version:

ComputerZen.com – Scott Hanselman’s Weblog – Will ANYTHING new ever work?

bq. There is a subtle (as a brick in the face) difference between “It just works” and “I got it to work.”

When I think about it, I realize how much this applies to the things I do on a day-to-day basis. Certainly most things in my life “just work”; cars, telephones (but not cell phones), kitchen stuff, light switches, hot water heater, laundry, furnace, and on and on. (Can you imagine what life would be like if they didn’t?)

And then there are computers (and other bits of computerised technology, like VCRs), where sometimes things “just work”, and sometimes “I got it to work”, and sometimes “I threw it out the window in disgust”. I’m often surprised by items in all three categories; sometimes things I expect will require a hammer and a lot of elbow grease “just work”, and (frustratingly) sometimes the opposite.

Where I work, we try very hard to make software that works, and yet are continually surprised by the bizarre things people do with their configurations before they call and complain. Reliability in the face of unexpected problems with computers and networks is one thing; reliability in the face of determined administrators is another thing entirely.

Anyway, I’m not sure where I’m going with this, so I’ll stop. Go read Scott’s article; I think it is interesting.

From Teal Sunglasses: It pays to pay well.. As usual Chuq nails several interesting ideas. But I particularly liked this quote, only slightly out of context:

bq. ask anyone who works for any company that builds things and deals with finances about the joy of figuring out who is responsible for warranty costs. Is it ever the group responsible for building reliability into the product in the first place? Nope. Want reliable products? Take warranty repair costs out of your development budgets, not your support budgets. and watch your development managers have big, ugly, purple cows…

Ouch, says I, wearing my CPE hat! CPE -> Current Product Engineering; my team fixes software that has already been released to customers. Software is different than hardware; repairing or replacing hardware is more expensive than patching software. Anyway, the costs for my team _do_ come out of the development budget. We also have a separate customer support hierarchy, responsible for all of the day-to-day handholding that customers need. Theoretically problems don’t cross into my domain until they are confirmed defects (as opposed to “controlled flight into terrain”), but in practice we end up debugging the hard problems on both sides of that line.

Anyway, it’s certainly fun and challenging work…

I remember the days when “rms”:http://www.stallman.org/ refused to port “Emacs”:http://www.gnu.org/software/emacs/emacs.html to “Windows”:http://www.microsoft.com/ (or the “Macintosh”:http://www.apple.com/) because they were closed platforms and violated the spirit of free software.

Today I have “Emacs”:http://www.gnu.org/software/emacs/windows/ntemacs.html, “Apache2”:http://httpd.apache.org/docs/current/platform/windows.html, “PHP”:http://www.php.net/manual/en/install.windows.php, “Perl”:http://www.activestate.com/activeperl, “Python”:http://www.activestate.com/activepython, “MySQL”:http://dev.mysql.com/doc/refman/5.7/en/windows-installation.html, and “Firefox”:https://www.mozilla.org/en-US/firefox/new/ all running on my spiffy new Windows XP based laptop…

I feel so un-pure :-)

Why do you have to click the Start button to shut down?

bq. That’s when we decided to label the System button “Start”.

bq. It says, “You dummy. Click here.” And it sent our usability numbers through the roof, because all of a sudden, people knew what to click when they wanted to do something.

The real science of usability… (laughter)

Critical Section – The Tyranny of Email

bq. Email is one of the greatest things the computer revolution has done for personal productivity. Used improperly, it can also hurt your productivity. This article discusses ways to use email effectively. Then it goes beyond that and talks about how to be productive, period.

Another get_weather_ca update for misterhouse. Environment Canada reformatted their text-only web page again:

* they changed a bunch of ULs to DLs.
* They re-ordered the sections on the page (messing up the forecast finder).
* they added spaces _before_ all of the colons.

I also added a “[Read,Show] weather forecast” to my code/weather_ec.pl code, and fixed a minor bug in the dewpoint calculator. Of course, that code only gets triggered when something is broken these days, because EC now publishes the dew point as part of the data.

The patch is against 2.78; I haven’t upgraded to 2.79 yet. (It figures that as soon as 2.79 was released, my code would be obsoleted again :-)

You can download:

* “patch against misterhouse 2.78”:http://www.cfrq.net/~chk/misterhouse/2.78/get_weather_ca.patch
* “weather_ca.tar.gz”:http://www.cfrq.net/~chk/misterhouse/2.78/weather_ca.tar.gz
* “weather_ca.zip”:http://www.cfrq.net/~chk/misterhouse/2.78/weather_ca.zip

It turns out that as part of the “website revamp”:http://weatheroffice.ec.gc.ca/mainmenu/whats_new_e.html, there’s now a text weather page, as well as the fancy one with tables and graphics. Actually, I tripped over it by accident; on the weekend the graphics page was redirecting to the text-only page for some reason. When the graphics page returned, I noticed the “graphics off” link that gives the text page :-)

Anyway, on the weekend I wrote a parser for the text-only page. It seems to be working OK, so I cleaned it up a bit and am posting it here (and to the mailing list). As an added bonus, forecasts are back (yay!).

You can download:

* “patch against misterhouse 2.78”:http://www.cfrq.net/~chk/misterhouse/2.78/get_weather_ca.patch
* “weather_ca.tar.gz”:http://www.cfrq.net/~chk/misterhouse/2.78/weather_ca.tar.gz
* “weather_ca.zip”:http://www.cfrq.net/~chk/misterhouse/2.78/weather_ca.zip

Our friends at “weatheroffice.ec.gc.ca”:http://weatheroffice.ec.gc.ca/ have moved their scripts around, and changed the web page formats. I’ve partially updated my screen scraper for “Misterhouse”:http://www.misterhouse.net/, although I’ve had to disable the forecast portion until I get more time.

You can download:

* a “patch against misterhouse 2.78”:http://www.cfrq.net/~chk/misterhouse/2.78/get_weather_ca.patch
* “weather_ca.tar.gz”:http://www.cfrq.net/~chk/misterhouse/2.78/weather_ca.tar.gz
* “weather_ca.zip”:http://www.cfrq.net/~chk/misterhouse/2.78/weather_ca.zip

I found an article: Whole Earth: Dancing with Systems

self-organizing, nonlinear, feedback systems are inherently unpredictable. They are not controllable. They are understandable only in the most general way. The goal of foreseeing the future exactly and preparing for it perfectly is unrealizable. The idea of making a complex system do just what you want it to do can be achieved only temporarily, at best. We can never fully understand our world, not in the way our reductionistic science has led us to expect. Our science itself, from quantum theory to the mathematics of chaos, leads us into irreducible uncertainty. For any objective other than the most trivial, we can’t optimize; we don’t even know what to optimize. We can’t keep track of everything. We can’t find a proper, sustainable relationship to nature, each other, or the institutions we create, if we try to do it from the role of omniscient conqueror.

Systems thinking leads to another conclusion, however – waiting, shining, obvious as soon as we stop being blinded by the illusion of control. It says that there is plenty to do, of a different sort of “doing.” The future can’t be predicted, but it can be envisioned and brought lovingly into being. Systems can’t be controlled, but they can be designed and redesigned. We can’t surge forward with certainty into a world of no surprises, but we can expect surprises and learn from them and even profit from them. We can’t impose our will upon a system. We can listen to what the system tells us, and discover how its properties and our values can work together to bring forth something much better than could ever be produced by our will alone.

We can’t control systems or figure them out. But we can dance with them!

This article sums up, in a nice way, what I do for a living. Cool.

From an article by Donella Meadows, published in Whole Earth Winter 2001, via heavybit.com.