November 21, 2001

6:19 AM

Software projects are notorious for time and budget overruns (examples that come to mind include Mozilla and the Denver Airport baggage system). There are a large number of design methods, development processes, and programming methodologies that claim or hint at objective estimation of development schedules, project complexity, and programmer productivity. Unfortunately, they're all bunk.

"The creation of genuinely new software has far more in common with developing a new theory of physics than it does with producing cars or watches on an assembly line."

Programmers, try telling that one to your next customer.
posted by lagado (21 comments total)

All software professionals know this; one wag suggested that "Computer Science" really should have been named "Computer Craft".

But it's also well known among software professionals that the primary source of overruns is feature creep. That's what we need to be communicating to everyone else, and so far we haven't done a good job of it. Software projects run a lot better when everyone agrees ahead of time what the goal should be, and then sticks to it. (One project I helped run came in 3 weeks ahead of schedule with a very low bug rate, precisely because the project manager was a software guy and didn't permit feature creep.)
posted by Steven Den Beste at 7:03 AM on November 21, 2001

Man, do I agree with you, Steven. I had two truly outstanding, slam dunk projects where I last was. In both cases I managed to produce a highly detailed, pre-coding, requirements document that changed little over the course of the project. On time, on budget, satisfied team and satisfied clients.

lagado, one suggestion: if a link is to a PDF, I think it's worth noting so.
posted by mmarcos at 7:19 AM on November 21, 2001

mmarcos: outstanding. how did you keep the clients from throwing your document in the trash and giving you one written with crayons two weeks before it was due?
posted by lescour at 7:21 AM on November 21, 2001

Project planning and estimation is an art. After writing a vast amount of code and fixing a sea of bugs (yours and others), you start to get a feel for how long certain tasks will take and what sort of bugs will result.

What I end up doing is look at a design and break it into components and then estimate how much time each task will take me to do along with some SWAG (Silly Wild-Ass Guess) time. If the tasks are well-defined, then the smallest unit of estimation is a 1/2 day. The less well-defined the tasks, the larger the granularity. Then for very large projects, you need to start adding in time for human problems. In a year, you can expect to lose up to a month of productive time per engineer. It can be for things like illness/infirmity, family issues, etc.

Then I start estimating complexity and of the system and start guessing how many bugs will result from each in terms of simple, moderate, and bad. A simple bug is easy to find, easy to fix. A bad bug is a one that either takes a very long time to find, a very long time to fix (lots of ripple), or requires redesign or rearchitecture. Moderate bugs are in the middle.

Then I scale the tasks and bugs by a factor related to the person who will be responsible for that task as well as their familiarity with the task. If it's me, the factor is 1. For others, I get a sense of the factor through experience (eg, I work 10 times as fast as this person or 1/2 as fast as that person).

The estimates are hauntingly accurate for well-defined projects.

Where problems arise is when the project staff or executive staff doesn't pay attention. The last place I worked, they had a date they needed to meet for a marketing event, and I gave them a time estimate for implementation and an estimate for when I absolutely positively had to have UI design in my hands. UI design was late and the project was late by exactly the same amount. This was consistent enough that when I was asked, "How long will this take?" I wanted to start answering "Will you believe me this time?"
posted by plinth at 8:24 AM on November 21, 2001

My manager's method is a bit more simple, but involves killing a goat and reading the fortune from its guts. :-)

Actually, I've been reading a lot about project management and specially software development projects.

There are some interesting ideas about it, such as Eli Goldratt's Theory of Constraints applied to Project Management (more about it on some of his books, such as
The Goal and Critical Chain).

Also, a friend of mine is managing a small team that is responsible for a big software development project and he told me they are using some ideas of Extreme Programming.
posted by rexgregbr at 9:07 AM on November 21, 2001

Plinth, SWAG stands for "Scientific Wild Ass Guess".
posted by Steven Den Beste at 9:37 AM on November 21, 2001

Yeah, I can totally believe this. After a dozen years writing code I'm pretty good at estimating the amount of time needed to write something similar to something I've written before. I know the territory and have a reasonable idea what sorts of problems are likely to come up when traversing it. But if any significant amount of the problem involves systems or tasks I've never encountered before, I pretty much just need to guess. This seems to be common - part of making an estimate seems to involve figuring out how much of the project is unknown, and tripling the amount of time it looks like it will require.

About three months ago I completed a project in 15% of the time I had estimated - this certainly isn't the first time in my career I've been wrong by that much, but I think it will probably be the only time I've ever been wrong by that much in that direction.

-Mars
posted by Mars Saxman at 9:39 AM on November 21, 2001

One rule of thumb I read one time, which is correct a scary amount of the time, is to ask the programmers how long something will take, double it, and raise it to the next higher unit of measure. If he says "1 day" it will take 2 weeks.
posted by Steven Den Beste at 11:39 AM on November 21, 2001

CEO: "Ok, we've agreed that the project will take 5 months to complete so we're going to estimate that we have 7 months"
Engineering team: "cool"

6 months later...

Salesguy: "I just promised our biggest client that the product will include ponies. I said we could do it. I didn't bother to ask anyone if we could, because I figure you guys are smart"

CEO: "You guys can add ponies, right?"
Engineering Team: "Well, we could but..."
Salesguy: "So we'll expect ponies. Bye, gotta go drink... er do research"

8 months later

CEO: "You guys are late and there aren't any ponies. In punishment for this, I'm going to let go half of your team"
Engineering Team - 1/2: "But..."
Salesguy: "I just promised daffodils to another client"
CEO: "Well, the half of you will have to work twice as fast"

Repeat ad infinitum...
posted by owillis at 11:49 AM on November 21, 2001

Oliver, I'm getting nightmares...
posted by Steven Den Beste at 11:57 AM on November 21, 2001

Nightmares? Hell, I didn't even add the client's request for modifications after delivery...
posted by owillis at 12:02 PM on November 21, 2001

In the older project, it happened to be somewhat under the covers. My idea happened to cost very little to implement on existing software and infrastructure, found tremendous support from a key developer and key manager, and the requirements were accepted or denied by the manager who was a real toughy. I got everything from him before coding began because he knew exactly what he wanted and he wanted it quick with no fancy stuff.

For the later project, I worked on convincing the manager that the requirements would help for the obvious reasons, etc. and found a good partner there, too. In this case, the manager did ask for some later things, but they were more of the tweak nature or maybe a little more sometimes, but this was built into the project timeline.

In spite of the success, the organization did not change much in this respect, unfortunately.
posted by mmarcos at 12:11 PM on November 21, 2001

But it's also well known among software professionals that the primary source of overruns is feature creep.

That's true, although the main point of the linked (PDF) paper was to point out that even with a complete functional specification, software estimation methods (i.e. algorithms) run into fundamental mathematical limits of computability and completeness (based on the work of Godel Turing Chaitin).

The crux is that any estimation methodology that claims to be objective rather than just a rule of thumb or a guesstimation is blowing smoke rings.
posted by lagado at 3:58 PM on November 21, 2001

(based on the work of Godel Turing Chaitin)

God, imagine growing up with a name like that!
posted by kindall at 4:39 PM on November 21, 2001

ha! kindall ;-j sorry, forgot to comma delimit those.

Here's a brief introduction to the ideas behind Chaitin's Incompleteness theorem.
posted by lagado at 6:47 PM on November 21, 2001

My last big corporate job was as project manager for a web-mail app that needed customising, internationalising and rolling out across local affiliates with very different setups and very independent-minded sysadmins. It was quite an experience, ranging from reverse-engineering a badly coded Java engine, to documenting how to sweet-talk the most stubborn European office. (The Austrians. Always, always the Austrians.) It was the kind of project that had so many barely-tangible human elements that it defied even the most precise speccing, which meant I was squeezed on one side by my team's frustrations, and the other side by the ultra-efficient American MBAs upstairs with their nice little fictional plans as generated by Microsoft Project. Let's just say that Frederick Brooks became my hero.
posted by holgate at 8:07 PM on November 21, 2001

ultra-efficient American MBAs upstairs with their nice little fictional plans as generated by Microsoft Project

hey, those fictions are the only thing that gets money budgeted. I'm sure they'd let non-Americans play with MS Project, too, if there weren't money involved.... ;)
posted by mattpfeff at 9:06 PM on November 21, 2001

I'm actually with Joel Spolsky, though: MS Project is completely unsuited to software projects:

Don't use anything fancy like Microsoft Project. The trouble with Microsoft Project is that it assumes that you want to spend a lot of time worrying about dependencies. A dependency is when you have two tasks, one of which must be completed before the next one can begin. I've found that with software, the dependencies are so obvious that it's just not worth the effort to formally keep track of them.

So while wearing my "talking to the MBAs" hat, I'd smile and adjust the fictional MS Project plan to keep them happy (and keep my team funded), while doing the real planning in Excel.
posted by holgate at 6:56 AM on November 22, 2001

You mean you guys plan your software? That makes it even worse ;)
posted by walrus at 8:04 AM on November 22, 2001

real planning

heh. Who's talking about fictions now? ;)

(I can see it now, the MBAs saying, so while wearing my "talking to the PM" hat, I jus' smile and tell 'em I'll adjust the MS Project plan, to keep 'em happy.... Of course, now that I think about it, that's why the MBAs (almost) all suck -- sometimes the PM actually knows wtf he's talking about.)
posted by mattpfeff at 8:38 AM on November 22, 2001

It's fictions all the way down.
Imagine the porkies they tell holgate just to keep his spreadsheet happy
;-j
posted by lagado at 3:46 PM on November 22, 2001