We had debates over the many months of development of Laszlo Mail about whether we should just go ahead and post the app so people could use it, rather than focusing on developing the licensed version for customer X. I feared that it would be distraction and the next deadline for a paying customer was perpetually looming.

Sometime last summer, I started to seriously schedule the work items associated with a hosted offering (not a huge number of things, but different from what is required for licensed software). Our ability to execute on these by the end of the year seemed bleak. There were requirements and feature requests from customers X, Y, and Z. Nonetheless the marketing department wanted a splash for the Web 2.0 conference and the unlikely end-of-year launch date was pulled in by a couple of months with just a little re-definition of what requirements for a public launch. I steeled myself for a deluge of snarky requests by grumpy people who have high expectations for free stuff.

After a brief “friends and family” launch, we went live in early November. Contrary to my expectation, people were very nice. Even those who reported bugs took the time to point out things that they liked about the software. On the whole, bug reports were clearly written with reproducible steps. Suddenly, it was much easier to prioritize the bug list. It’s also way more fun to fix a bug when you know that someone personally cares about it. It’s not just a checklist item. It’s the bug that Ron found, or the thing that really affects Kent and Amy’s ability to read their mail. (We were also “eating our own dog food.” Many Laszlovians were using Laszlo Mail as their primary way of reading email, and everyone was using it for webmail.) The launching of laszlomail.com was a distraction, but one that was well worth it in providing the addition of direct feedback from real people.

It is worthwhile to note how we have changed our development methodology to make our hosted offering an effective distraction. We follow a number of guidelines contributed by various members of the team.

• “never release on a Friday” — Scott’s rule, and generally a good idea. I don’t think anyone should work a weekend unless they really want to or it’s really an emergency. No need to manufacture emergencies.
• “just ship it” — Temkin’s rule, which is oddly juxtoposed to his role as the feature creeep. Sometimes a new feature is worth imperfect behavior in another area. No one plans it that way, but anytime you change code, you risk introducing bugs. In the drive toward creating a vibrant, frequently updated application, most have us have needed to change our definition of a “stop-ship” bug.

A consistent and vigilent use of our bug database contributes to our ability to iterate quickly. I’ve shipped lots of software, but my bug-scrubbing skills were most effectively honed in shipping Macromedia Director versions 6, 7, and 8. Since then, I’ve carried with me the following bug priorities decriptions, and it is their very detail which leads to their effectiveness:

• P0: blocker. Drop everything and fix this bug. This means the build is broken or someone can’t do their work because of this bug. These bugs are rarely open for more than 24 hours and there should never be two of these assigned to one person. If there is a P0 open, it is all-hands-on-deck, and whoever would be most effective at fixing this bug should be on it.
• P1: stop-ship. This bug alone would cause you to make a stop-ship decision. In fact, if you discovered this bug right after burning the final CD, you would run screaming after the truck and yank the package out of the hands of a startled driver, so as to prevent it from going to manufacturing. Tanslated into web terms, this is not so dramatic, so I find the old definition still helpful.
• P2: must-fix. You plan to fix all the P2s, but it would take a collection of these to cause a stop-ship decision. You want to have zero P2s at some point, but generally you’ll find more P2s before you ship, and you’ll ship anyhow.
• P3: plan-to-fix. You want to fix all P3s, but you usually don’t. There can be any number of these open when you ship. These are real bugs that will happen to real people, but they are usually edge conditions and minor inconveniences.
• P4: may-never-fix. Bugs which are hard to reproduce, of arguable merit, or hardly noticeable live here. I try to avoid using the term “cosmetic” since aesthetics do matter. Viusal consistency contributes to a feeling of quality and there can be cosmetic P1s.

These priority definitions are absolutely subjective. Over time our team has developed a good group-think around priorities and while we often discuss them, there is rarely great disagreement. Of course, engineers do have the freedom to fix any bug at any time, and can go ahead and fix a P3 before a P1, if they feel strongly about it or are in the neighborhood of the code anyhow. The people who write the code are the ones with the real power. That’s why we make sure to hire great people.

Priorities are always relative to a specific milestone. It was remarkable to me that our recent release to laszlomail.com exhibited some rather unfortunate font inconsistency. In a licensed software release, that kind of bug would have been in my cosmetic-P1 category. It’s those kind of bugs that can make software look shoddy and not well put-together, challenging your faith that the software will work consistently in other ways. However, if you are releasing live software, that is expected to grow and change every week or two, perhaps expectations can change, at least we judged it so. I’m hoping that people will consider it more of a bad-hair-day for the application, rather than a systemic deficiency. Better to go out before the holidays with its new features in tow, rather than stay home until it can come out well-coiffed two weeks later.

This entry was posted on Thursday, December 29th, 2005 at 1:26 pm. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.