speed vs. size

I was doing wall-clock timings of how long it takes to launch Firefox vs. IE a few days ago, and was surprised to find out that IE is now much much faster.

When Firefox 1.0 came out, the faster launch was a big notable thing. IE was bloated, the argument went, and so you should use Firefox instead.

But the trouble is that added features and team size actually work against you on this kind of thing. If any single checkin can break app startup time, then there are many ways for things to go wrong, and only a few ways for them to go right. Same thing with features. An "important" feature can attract a lot of attention, but it's not always clear how it interrelates with overall application performance.

We've had to build some sophisticated systems to have Picasa performance keep up as we add more background features and systems. e.g., we do a lot of work in the background, and the UI is generally as fast as Picasa 1, which did a lot less simultaneous work.

You should do performance regression tests with your app. In many ways, changes in top-line performance give you more insight than lower-level testing can. Done well, you can identify a particular changelist that caused a slowdown.

Making good tests like these, however, is very hard, and kind of ill-defined. For instance, Picasa shipped a couple versions with its image prefetcher totally broken (which makes everything feel slower), but since it only works in some circumstances anyway, nobody noticed. There wasn't a good test for it, and there still isn't.

But my advice is to measure your app's speed wherever you can, and pay attention to changes. You will find more bugs, too.