Knowing What To Count
From Programmer 97-things
"Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted" — Albert Einstein
Wherever you look, software development is measured. Whether it is number of requirements, lines of code, number of tests, code coverage, number of bugs – someone, somewhere is counting it.
Now I am the first to admit, that metrics can be very useful in monitoring progress and establishing the health of an application development. Many items can automatically be computed as part of the development cycle so that the effort in assembling the metrics is minimal. However, unless you know why you are collecting the data, and how to interpret the data, the effort is often wasted.
Lines of code (SLOC) appears an easy thing to measure. It is only when you look a bit deeper do you realise that there are some many different interpretations that it almost becomes a meaningless measure. Do you consider comments in the count? Is it executable statements that are of interest? I have seen SLOC used to determine how much more development is outstanding, not realising that 'good coding' practice may actually result in less lines of code (and the less code written, the less chance of a bug). Similarly when the code size exceeded the estimate by a considerable margin, the developers were asked to recode so that the code size was closer to the estimate (not realising that the code count was including a considerable amount of proven library code which had been used to deliver the application).
Asking to count the right things takes experience. I remember one colleague asking how many tests had been completed. When I responded 90%, he went away content, thinking it wouldn't take long to go through the remaining 10%. However, this was the wrong question. What he should have asked, was how many requirements I had passed (probably around 15% because the majority of the outstanding tests were to demonstrate the more difficult requirements).
In many cases, there is often a more meaningful metric to consider. For example
- Instead of Lines of Code, a better measure would be code coverage and complexity since this gives an indication of the amount of testing that has been done and the future maintainability of the code base
- Instead of number of Tests passed, a better measure would be number of requirements passed as this is a better indicator of progress
- Instead of number of bug reports, a better measure would be number of bug reports implemented and integrated into the code base.
Regardless of the metric, whatever is measured must be used to encourage good practice remembering that as the development moves forward, the metrics should change to reflect the changes from implementation into integration and deployment.
This work is licensed under a Creative Commons Attribution 3
