This post is the eleventh part of a ramble-rant about the software business. The current posts in this series are:
This post is the first installment in a mini-series about the fundamental nature of software development.
"Can Software Be Manufactured?"
I'm so glad you asked. Pull up a chair here beside Grandpa Andy and let me tell you a little story.
A long time ago, back when the years began with the number 1 and the US had a manufacturing economy, Quality became important in manufacturing. Quality was important to some enterprises before this time, but it became fashionable for lots of manufacturing enterprises later. After this, terms such as "six-sigma" and "ISO9000" were common.
In the US, manufacuring plants bought into Quality in a big way. There were classes, seminars, studies, and books - lots and lots of books - about applying the principles of Quality in every industry. I was working in manufacturing during this frenzy. I took the classes, attended the seminars, participated in the studies, and read a lot of the books.
I have a confession: I have always enjoyed Predictive Analytics.
Here, embedded in six-sigma and other Quality programs and procedures, were ways to predict stuff about the number of widgets produced and their quality. For an engineer who enjoyed predictive analytics, this was awesome! I really enjoyed using Quality processes to improve the performance of manufacturing processes.
Manufacturers thought "See these results? This is fantastic. Let's do this everywhere!"
So they implemented Process Metrics collection in maintenance departments and inventory management. And, one day, it was implemented in IT. A funny thing happened in IT: Quality programs didn't work. Why?
Software is organic. It starts small - germinates as an idea really - and then matures. It grows.
How do you measure it's progress? How do you measure anything organic - say, a tomato plant? If you're measuring a tomato plant you can measure its height and record that each day. Is this a valuable metric? What you really want to know is:
When will it produce fruit?
How much fruit will it produce?
How long will it produce fruit?
Will measuring the rate of growth tell us any of this? Not really. Measuring the plant height will tell us it's getting closer to producing tomatoes, so that's something. But since tomato plants of varying size produce fruit, this metric lacks confidence.
Back to software: What were the related metrics for software development in the manufacturing IT departments?
One way to develop software is to break it into three phases:
Counting lines of code could work in the Frame and Build phases. But in Optimize? A really good optimization may involve removing lines of code. So this metric backfires in a major way at a crucial time in the development lifecycle. Bug counts are not linear and therefore not a consistent metric for tracking development progress. Hitting an SLA or deliverable? This is a good metric, but it's only available at the end of the project.
If you're sold on measuring progress, what do you do?
Listen to the Developers
Software evolves throughout a project (it's alive, remember?). This is one reason Agile methodologies work: Agile accounts for the true nature of software development. And part of that nature does not lend itself to repeatable and deterministic measurement.
Software cannot be manufactured - at least not in 2010. This is why experts at manufacturing who were also experts at Quality could not make it work - even in their own IT departments.
But you can listen to your developers. Evidence-based approaches are a neat blend of metrics collection and paying attention to the developers. Consider a WBS (Work Breakdown Structure) approach to software - which works really well in other engineering disciplines, by the way. WBS is forward-looking, attempting to extrapolate the effort and duration required to build components of a software project. It's not so much extrapolation as it's addition. Breakdown the tasks into manageable chunks, estimate each chunk, then add up the numbers. Simple right? Too simple, in fact - but I'll get into that later in this mini-series.
Battle of the TLAs*: EBM vs. WBS
Contrast this with Evidence-Based Management (EBM). Like WBS, EBM breaks the software development into components and phases. Unlike WBS, the evidence-based approach relies on historical data gleaned from the first (or first few) development efforts. Effective metrics can be collected early in the development phase by allowing a developer to do what they do best - develop software - while measuring how long the development effort takes, in both effort and duration.
If want to get all crazy, you can ask the developer before she starts for estimates on effort and duration. Then, when that chunk of code is built, a good project manager can compare the estimates to the measurements. It's vital not to sabotage this cool start by holding the developer's feet to the fire. The phrase "But you said it would only take ___" should never cross your mind - much less be communicated to the developer. You're trying to build a performance characteristic here. What you want is a way to predict (as in predictive analytics) when the project will complete. Good project managers make this claim all the time: "I just want to know when it will be done. I don't care how long it takes. I need a number I can communicate with confidence to the client." If that's true, here's your chance to prove it. Collect the developer's estimate and then collect the evidence - the actual effort and duration ex post facto - then do the math. Developers are creatures of habit. Some will guess high, most will guess low, every now and then someone will nail it. It's a guess. If you pop enough of these metrics into statistics functions on your favorite calculator, you will see a pattern emerge. And this pattern can be a estimated:performance ratio you can begin to apply to each developer's estimates. (Like I said, crazy...)
The same characteristic exists for software teams. And tomato plants. For example, I know that if we wait a couple weeks beyond the recommended planting date for tomato plants in Farmville Virginia, we get more and better tomatoes out of our garden plot. Why? It could be any number of reasons really: The amount of sunlight that hits our garden, the soil temperature and composition, how we manage weeds and pests. I'm certain all that plays into the result, but the result is what I'm after - and that's more and better tomatoes. I don't need to be an agriculture major to grow tomatoes. I just have to experiment, measure, and record the results.
The tomato plants, like the software developers, know what they need to do. Our job is to make them as effective as possible.
*TLA is a three-letter acronym for Three-Letter Acronyms.