Measuring Developer Productivity

Posted by Sajith M on Jul 13th, 2007
2007
Jul 13

My job requires me to periodically submit information on the productivity of a software development team (team consisting of software developers), and more importantly this has to be a number and (however much I might like the non-measurable answers, they) can’t be like “doing good”, “better than last year” and things like that. Basically I need to quantify the productivity of a team or an individual. What is a good measure of productivity and how do we get to measuring it?

Who is a more productive developer - the one who writes the best code that don’t fit the requirements, or the one who writes mediocre code that fits the requirements? Or to ask the same question in a more buzzword rich language: What is more important - efficiency or effectiveness?

To begin with, what is productivity? Productivity can perhaps be best defined as “The amount of output per unit of input”. Hence, to measure the productivity of a software developer we need to measure the output of software development. And since we can’t measure the output of software development, we probably can’t measure software developer productivity.

Of course, that does not mean that people don’t try to measure developer productivity - they do and in a number of ways. The fact that none of them is a good way to measure productivity has not deterred people who believe that they need numbers to be in control, or to be knowledgeable about the progress being made…

The first and perhaps the most popular method is measuring the lines of code. The assumption being that the more lines of code you write, the more productive you are. The simplicity of this method is also its greatest weakness. As anyone who has written code for a living can attest, there are a huge number of ways in which the same problem can be solved and each of them results in a different number of Lines of code. Then there is the question of how the number of lines are calculated - are braces on a different line taken into account, what about empty lines, comments etc. Yet despite its obvious weaknesses and a complete lack of correlation with the productivity, it continues to be the most popular measure. Guess, that speaks something about out management techniques that depend so much on figures that it willing to risk its life on any methodology that comes it way, however irrelevant it may be.

LOC is not the only measure that is used to calculate productivity. Another common one is function points - the number of function points implemented by a developer in a given space of time. This is better than counting LOC but still pretty inaccurate if you consider the fact that function points for a given feature can vary by an order of magnitude depending on the method used to make the computation. When the productivity figures don’t depend on the developer and rather depends on who calculated the FP and using what method, we can be confident that we have a number that does not necessarily bear much resemblance to the actual productivity.

The third one, and relatively less used one is the bug count - the lower the bugs, the better you are. Of course, this is a valid measure of quality if everyone does same amount of work with the same amount of complexity. If you have ever done commercial programming - programming to earn a living, you would by now be laughing your lungs out just imagining everyone doing same amount of work and involving the same amount of complexity. The fact that people don’t do an equal amount of work, or work involving the same complexity renders the method pretty much useless to compare your programmers. Then of course, there is the fact that if you don’t write any code, you don’t have any bugs. Zero bugs means great productivity, while the actual productivity is a big zero. Me for one would not like to have this method used to compute productivity.

Another factor that we haven’t yet considered is that neither lines of code not function points actually measure the productivity of the software. For example which one would you consider more productive - a program that is 100 FP and 1000LOC and brings in $500,000 or another program that is 200 FP and 2500 LOC but brings in $100,000. And which developer would be considered more productive? Also, software does not provide returns immediately once it’s been deployed, rather the returns come over a period of time, and as such how do you measure the total productivity that is achieved by developing software?

The most common excuse presented in favour of the indispensible need for measure is “if you can’t measure it, you manage it”. Of course, having a good measure helps manage things, but saying that a measure is indispensible for managing an activity is a lie - a lie of the highest degree. After all, how are lawyers and creative people paid for their works?

If we could assess software much more easily and objectively than we can now, it would be great and perhaps a good measure could be worked out as well. But false measures only make things worse. I suspect that looking away from any agnostic, algorithmic solution and instead using own personal prejudices to judge how well the developers are solving the problems assigned to them would be a better bet. It boils down into personal judgement and actually having a clue of what the developers do, and measuring the wrong things is only making things worse. I think it’s better to accept our ignorance and stop measuring for irrelevant things than get the false measure and get false ideas based on the measures.

Of course, what do I report as the productivity of my team is still an open question. Any suggestions?

Update: anon points out This seems to border on plagiarism of this post by Martin Fowler: http://www.martinfowler.com/bliki/CannotMeasureProductivity.html. Thanks anon for pointing this out, I definitely did not want to copy Martin Fowler though I believe I did express the same thing that he had said.

Posted under: Technology/Software Development
Tagged with: , , , ,

Project Disaster

Posted by Sajith M on Nov 28th, 2006
2006
Nov 28

A project (software project) is a disaster in the making when:

  • you have a project that has vague requirements/unclear requirements
  • you have a project that has an impossible deadline and/or budget
  • you have a project that no one wants to be part of

One or more of these factors are a sign that you are on a project that is going the disaster way. These are three factors I could think of, what do you think?

PS: A blogger who happens to be on such a project might stop blogging for such a time, till the project gets over or is canceled (hint to Ekawaaz ;-)

Posted under: Technology/Software Development
Tagged with: , , , , , ,

Guide for managing Use Case (UML) driven projects

Posted by Sajith M on Sep 6th, 2006
2006
Sep 6

Jackie Hewett of Clarety Consulting has this guide for managing use cases

Posted under: Technology/Software Development

Threading in C#

Posted by Sajith M on Aug 24th, 2006
2006
Aug 24

Joseph Albahari has a great online tutorial on threading in C#, also available as a PDF document (direct link)

Posted under: Technology/Software Development

Not just fixing the bugs

Posted by Sajith M on Aug 23rd, 2006
2006
Aug 23

But how much work the software does is not what makes it remarkable. What makes it remarkable is how well the software works. This software never crashes. It never needs to be re-booted. This software is bug-free. It is perfect, as perfect as human beings have achieved. Consider these stats : the last three versions of the program — each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.

That was www.fastcompany.com on Software developed by NASA. The process emphasises not just fixing the bugs, but also finding the source of the bug and fixing the process so that the bug never gets repeated.

Posted under: Technology/Software Development

Personality Traits of the Best Software Developers

Posted by Sajith M on Aug 23rd, 2006
2006
Aug 23

Agreed. Read it here.

Posted under: Technology/Software Development

2006
Aug 3

Today I am (very much) in a linking mood. So here is a link to a post by Professor Sadagopan. Another shameless copy.

NR Narayana Murthy, Chairman, Infosys pointed out to the need for quality going beyond metrics and tools. In his talk on July 7, 2006

“Quality challenges for Indian software professionals”

at IIIT-B as part of the launch of South chapter of Bangalore SPIN (Software Process Improvement Network), he narrated the situation in 70’s and 80’s when typical Indian households were so proud of their Sony Trinitron TV at their homes, that they would not leave any opportunity to drive the conversation to TV, and make sure that the guests visiting home notice the fact that they are one of the proud owners of SONY Trinitron TV!

Murthy exhorted the Indian software professionals to aspire for the same position.

Will the companies who hire Indian software professionals / companies, do the same thing? Will they tell their friends with a sense of pride, that they got their software written by company x from India and they are damn sure it will work flawlessly day in and day out?

That alone will be a sure indication that Indian software has arrived.

Posted under: Technology/Software Development

? clear and concise code -> ternary operator

Posted by Sajith M on Jul 21st, 2006
2006
Jul 21

Was having a little discussion when someone said that the ternary operator should not be used because it is difficult to understand and maintain.

condition ? value if true : value if false
This is the syntax of a ternary operator. The condition is evaluated and if true the first value is returned, otherwise the second one is returned. What is so unmaintainable about this? What is difficult to understand about this?

Take this code for example. Can someone tell me what is difficult to understand or maintain
public String MyProperty {
get {
return (ViewState["MyProperty"] == null) ? String.Empty : ViewState["MyProperty"].ToString();
}
}

Or for that matter, how is this code better than the previous one
public String MyProperty {
get {
if (ViewState["MyProperty"] == null)
return String.Empty;
else
return ViewState["MyProperty"].ToString();
}
}

And yeah, I think ternary operator is simply beautiful so please don’t tell me not to use it.

Posted under: Technology/Software Development

How NOT to interview a developer

Posted by Sajith M on Jul 11th, 2006
2006
Jul 11

This is not strictly a follow-up to the other post I wrote - How to interview a developer. In my previous post I had written about the questions I usually ask candidates. In this I will describe what people have asked me (when I was being interviewed) that in my personal opinion are worthless questions and a complete waste of time.

  • How would you solve this problem? [The problem statement here]
    Some people think this is a great way to understand the candidate’s problem solving capabilities. The problem with this approach is that unless you problem statement is well defined (and I have never seen it that way), there are literally hundreds of right answers (and these need not be the ones anticipated by the person conducting the interview)
  • (take this paper) write some code to do [Some basic functionality]
    Since when did people start writing code on paper using pen. If you want to see if a candidate can write code or not, give him a laptop with the development environment and see if he can write code. People in real world write code using their favorite IDE that features auto-complete, syntax-highlighting, API help and everything else that people usually use. People in real world do not write code on paper using a pen, so please don’t test for it.

Am sure someone could justify both these practices. But to me they are a strict no…

Posted under: Technology/Software Development

How to interview a developer

Posted by Sajith M on Jul 11th, 2006
2006
Jul 11

Having interviewed many candidates for developer position at Stylus Systems Pvt Ltd, I think its probably worthwhile to discuss the questions I use to evaluate a candidate. Someone might find it useful, or you may be able to give me tips on how to do things better.

  • How many languages do you know? Which is your favorite?
    The more languages the candidate knows, the better. There usually are always a few that both of us know and I tend to quiz the candidate about it - what you like, what you don’t etc.
    The favorite is usually C#/Java/C++ (95% or more time), and I tend to focus on this and see how well the candidate knows his favorite language.
  • Which was the project, you learned the most from? Or Which was your toughest project?
    In both cases, a description of the project is asked for and also why it was a learning experience or why it was tough. While explaining the project, I tend to interrupt and ask specific questions about the project (exactly how or why was a particular functionality implemented) and the answer tells if the candidate knows what he is talking about.
  • When would you call a function too big? How big a function is really big?
    Anything more than a page of print is way too big and should possible be broken down.

These are some of the questions I ask to gauge the candidate. I would love to hear from you about your favorite questions.

Posted under: Technology/Software Development

Next »