Michael Wolfe on Quora posted an amazing analogy explaining why software development estimates are routinely way off, which results in developers padding their estimates to factor in the fudge of such inaccuracies.

Another thesis I’ve been pondering over lately are the terms “Software Engineering” and “Computer Science”, which imply that software development is purely scientific and engineering in nature. Most colleges run those programs out of their science or engineering departments, and the business surrounding technology (budgets, management, and processes) is premised on the assumption that coding is engineering in that it’s scientific, mathematical, calculate-able, and algorithmic.

All of that is a component for sure – but the process itself, the act of coding is much closer to art. You start off with an idea of what you need to do and how you’re going to do it, you start roughly putting it together, you realize later how things will really need to be put together so you make alterations/refactor, you get feedback from QA testers/analysts/focus groups, and continually refine until you get the final product.

So unless you’re solving the exact same problem every time, what exactly are we basing the effort on? If a carpet guy knows he can carpet a 2 bedroom house in a day, he can be very close with an estimate on a 4 bedroom house. And overtime, the variation of houses is fairly limited.

However, every software project is a new problem; why would the business ask to solve the same problem twice? So we’re estimating based off of as-similar projects done in the past, but unless it is purely a CRUD with zero rules, it’s unique.

Of course an estimate will always be an estimate. But plans & dates get created, promises are made around those estimates, and the estimates become commitments. Which then causes developers to factor in some fudge knowing that the estimate will become a commitment.

Would it help if we get rid of the word estimate and label that value for what it really is, the guess? Imagine updating all your change tracking, ticketing systems, and project management tools to replace the “Estimate” field with the term “Guess”? It would help manage some expectations as to what that number really represents.

The flip side

The problem on the flip side is that businesses are run by plans. They’re not designed to function off of guesses. For example Apple has to gets the contracts signed and dates sealed for the Apple WWDC, payments have to be made to Moscone, Moscone has to prep the building, Apple has to build their booths, sign up all the exhibitors, attendees, etc… so whatever software is planned to be presented has to be ready to roll by then.

It’s a fascinating challenge, and here are some thoughts.

1. Break down the problem into a collection of micro-small problems.
• Smaller problems are easier to size, internalize, and visualize. Many Agile based processes employ this in the form of small as possible user stories.
2. Get formulaic about it.
• If a team consistently applies a 2X b.s. factor, and is 80% accurate – you then have variables to build a formula & pattern around it where you project a target date -/+ x% based on the accuracy, and then build in risk mitigation strategies to make sure you’re managing the risk of not being accurate.
3. Prioritize & focus.
• A good practice at any level (from executive to individual contributor) at any thing (business, coding, personal life).
• Make sure you focus on achieving at least the top priority in it’s entirety by the target date instead of having advanced multiple priorities but never fully completing any. One thing done is better then 10 things almost-done, because almost-done is the same at not done.