And yet, here I am with Hummingbird, which is totally dependent on Twitter’s bandwidth. (In my own defense, I can only point out that: a) I wrote it because I needed the functionality; and b) I’m not building a money-making company around Hummingbird. I’m just giving it away.)

In the past couple of weeks, I’ve noticed that Twitter can sometimes take an astonishingly long time to provide an update. Hummingbird works by requesting URLs like http://twitter.com/statuses/user_timeline/kmactane.xml?count=25, and (up until now) it worked on the assumption that Twitter couldn’t possibly take longer than 10 or 15 seconds to respond to such a request.

That turns out not to be the case.

In fact, my recent tests have shown that Twitter can sometimes take over 5 full minutes to finish responding to such a request. This is a problem, because the default installation of Hummingbird tries to update its cached data once every 5 minutes. And since I assumed the request would be fulfilled in, at most, 10% of that time, I didn’t bother building in any concurrency checking.

I’ve just completely reorganized most of Hummingbird’s architecture. The interface that a blog owner sees (in the sense of the configuration variables in hummingbird.php, and the calling syntax for blog pages and cron jobs) is still the same. However, all of the code for retrieving data from Twitter has been spun off into a new hummingbird-cache.php file, which can be launched into the background by the rest of Hummingbird, so that it can patiently take as long as it needs to in order to update the blog’s tweets cache.

The Major Change This Causes

In Hummingbird’s previous incarnations, it would freshen an out-of-date cache before displaying it. The assumption was that every once in a while, someone might have to wait a few more seconds before seeing your blog page. However, everyone would always see an up-to-date record of your tweets. (Where “up-to-date” means “no more than 5 minutes old, absolute tops”. If you tweet so relentlessly that that’s a problem, you probably don’t have enough time to keep up your blog…)

Now that we know that Twitter can take over 5 minutes to give you the data needed to freshen your tweets cache, that design is not acceptable. Instead, Hummingbird is now committed to showing the contents of your tweets cache as quickly as possible, and only then does it fork the cache-update process into the background.

This means that, if you’re not using cron or some other automatic job-scheduling facility to run Hummingbird every 5 minutes, visitors who arrive at your blog after a period of inactivity will see an out-of-date, stale listing of your tweets.

If you have a cron job set to keep Hummingbird fresh, you’ll be fine. (A page view will also trigger an update, so if you get so much traffic that there’s never a 5-minute period during which your page doesn’t get hit, you’ll also be fine.)

Assumptions

1. You have a web-accessible server where you put your development .ipk files.
2. You can ssh into your Prē.

Download

You can download the script at http://kai.mactane.org/software/libraries/download/homebrew.sh.

Installation

Just drop the shell script into your home directory and make it executable. If you want to be able to use the "my" argument, you'll also need to edit the three configuration variables at the beginning: MY_HOSTNAME, STDPATH, and STDVERSION.

Usage

For most purposes, you'll just want to install your own package that you've been working on. Assuming you've set the config variables correctly, you can just type ./homebrew.sh my appname, where the appname is the base name of your package.

If you're installing multiple .ipks and you don't need to restart the GUI manager for each one, you can use the "skip" command on all but the last:

./homebrew.sh skip my foo
./homebrew.sh skip my bar
./homebrew.sh skip my baz
./homebrew.sh my quux

You can also supply a complete URL: ./homebrew.sh http://forums.precentral.net/spe_attachment/download-23971-com.palm.net.precoder.fcoaster_1.0.2_all.ipk will download and install fileCoaster, so you won't have to mess with my script any more.

What It Does

It automates the process of installing homebrew apps with a rooted Prē, as described on the webOS Internals wiki. Basically, it remounts the root partition in read-write mode, wgets your .ipk file, installs it, does the "Good Housekeeping" backup, remounts the root partition in read-only mode again, and then restarts the GUI manager.

I’ve also got a reasonably good script for installing, updating, and uninstalling homebrew apps for the Prē. Instead of the annoying, six-step process for installing homebrew apps on a rooted Prē, I just shell in and type ./homebrew.sh my 8ball, and the homebrew.sh script does it all for me. I need to publish that thing, now that I’ve got it working fairly well.

Additionally, my Japanese sentence generator, called “J-Babble”, now does proper plain past tenses (the -ta and -nakatta forms), which will make it more useful for me as a tool to keep me from backsliding when I’m busy. I’d link to that, but it’s not really a general-use tool yet. It’s more just for me. Maybe some day, I’ll give it the option for people to customize what vocabulary and grammatical forms they know, so it can just generate stuff they have a chance of understanding. For now, though, its use is just for me: when my life gets too busy for me to read my Japanese textbook and try to make new progress, I can at least bring up J-Babble once a day and get 25 randomly-generated, but grammatically correct and semantically sensible, sentences in Japanese. It’s just enough to keep the neural pathways from atrophying; it allows me to hold my place instead of losing ground.

(I’ve gotten some housework done, too, but this isn’t the place to talk about that.)

Well, now it does.

The change is pretty minimal, but it also incurred some overhead in my web site: the Hummingbird web page needed to be updated to match the new feature. And since that feature meant adding a new CSS class, I had to update the “CSS Styling” section as well as the “Features” section… and add a new “Version History” section…

All in all, it was harder to update the documentary web page than to update the software itself. In a way, that implies that my code was pretty clean. If you want, you can download the new version here.

See, I’m not neglecting everything to work on Palm Prē programming! (I’m also doing some contracting, which is taking up even more time.)

However, Livejournal is very customizable. It has 31 different “layouts”, each of which can then be further “themed” by application of CSS. This means that a user viewing a given journal, community, or single post, at a given time, may receive an HTML document containing any of 31 different DOM structures.

This means that finding a post’s author, or title, or even just a single post or comment, is not a straightforward process. I’ve set myself up a Ruby script that downloaded my “friends” view, a representative entry with lots of comments, and my own journal view, thus giving me a set of 93 fixtures that I can test against. The Ruby script also tweaked each fixture in the process of spooling it to my hard drive, by adding the proper calls to JSUnit files.

At the end of each document’s <head> element, there’s a call to jsUnitCore.js. Then, at the end of each <body>, I add a call to the lj-content-sieve.user.js script itself, as well as a set of test files that depends on which fixture this is. Every fixture gets an ljcs-global-tests.js call added to it — that file contains tests that should work anywhere, regardless of what sort of page you’re on.

Then all the “friends” page fixtures get an ljcs-friends-tests.js file, which tests operations that should happen on every friends page. For example, determining which entries need to be deleted. (In contrast, single-entry pages get ljcs-entry-tests.js, and the page from my own journal — which stands in for a view of a community — gets ljcs-self-tests.js.)

Finally, each fixture page gets a test file based on its layout: the “3 Column” layout gets ljcs-3 Column-tests.js, while “Cuteness Attack” gets ljcs-Cuteness Attack-tests.js. (Hey, I didn’t write or name the layouts; I just have to make sure LJCS works with all of ‘em.) These files will test that the actual DOM manipulations work properly.

Without test-driven development and automated testing to ensure that each layout and page-type is being handled properly, I don’t think this project would be manageable at all.

Do I really want to just write the whole thing, and then start profiling it to see where the hot spots are, and then possibly have to re-design the whole thing? That seems like the complete opposite of “work smarter, not harder”. Then again, it doesn’t matter if you write an O(n²) or even O(n!) algorithm if n is always going to be small… and in this application, I expect low-to-middling n values.

Of course, even if n will always be small, and increasing CPU power is my friend… even if it performs fast enough to make users happy, I’ll still know there’s a problem down there in the details. That may be the deciding factor.

The story is simple: I was tired of seeing “failed password” messages from sshd cluttering up my logs. I was also annoyed at the constant flow of dictionary attacks, even if I knew they’d never get in. So I whipped up a quick Perl script that acted as some glue between Swatch and iptables, and which would also give me some amount of reporting and history on who and what it was blocking.

Then I posted about it in my online journal, and a friend said it sounded useful. So I started getting it ready for release as a package that anyone could use…

And promptly realized that doing a decent, professional job of it would take more time than I had available. Fast-forward to now, when I’m unemployed and can only spend so many hours per day job-hunting — the result is that the world gets more software!

Now there’s GPLv3, and I’m really uncertain whether I like it at all. I understand that Tivoization is a serious problem, but I don’t quite understand the special non-consumer exemption for the anti-Tivoization clause. There are some other new and weird clauses the FSF has added, dealing with patents, and the WIPO Copyright treaty of 1996, and so on.

The GPL used to be pretty simple and elegant. Now? I recognize that the FSF is trying to make the best of a pretty bad situation. But their license has become something bizarre and awkward.

Theoretically, I could still use the old version. But that feels kind of odd, too, now that version 3 is out.

So now I have to look at things like the CPL, the Apache License, and so on. I’m already fairly sure I don’t want to use the MIT or modified BSD licenses; one of the things I liked about the GPL is that it keeps other people from dragging my open-source work into a proprietary package. I’m releasing it as open source for a reason, after all.

All of this is merely by of saying: I’m getting ready to release another software package. I’m not so sure the Creative Commons license I used on Hummingbird is quite as appropriate for this package, so I’m having to do some annoying research.

So I decided to get it cleaned up and make a real, distributable software package out of it. That took a few days, especially when I started banging on it, and trying to ensure that it had a prayer of working on anyone else’s server. Plus there was the time involved for writing the docs.

But now that all of that stuff is done, I am pleased to announce: Hummingbird. A lightweight, pretty, and compact gizmo for displaying your latest tweets on a blog or other PHP-powered web page.

* And if you don’t read Coyote Tracks… what the heck is wrong with you? *g*

It’s really tempting to completely refactor some of that stuff, but I haven’t got the time. I actually have at least one more software project that I should really package up and publish — a thing that might be helpful for system administrators who are being annoyed by SSH dictionary attacks.

Writing something that works on your own machine is one thing. Making a full release? That’s quite different! It needs documentation, and that documentation needs to explain all the things that you “just know”. It needs some kind of install mechanism, or else it needs to explain how to install the stuff manually — and you need to make absolutely sure that you haven’t forgotten any of the things you wrote, changed, or tweaked while you were making it work!