For the last few weeks I’ve been involved with helping a start-up get moving. You may have noticed that I’ve been a little fixated on tools for the last few posts and that’s why. I’m currently choosing tools and procedures, which is a nice change from having them foisted on you, and I want to be sure to avoid the mistakes I’ve seen in the past.
The reason tool choice is important is becasue your tools define metrics you can collect. For this post I will focus only on the kind of metrics you might use when managing a Agile/Scrum team or teams. But the concepts are more generally applicable.
The internet is abound with stories of bad metric choice, setting pay rises based on the number of lines developers checked in is just one example. I’ll leave that kind of metric for a later post, if I ever address them at all.
But that example of a bad metric does serve to introduce some general principles for metrics.
I’ll discuss each of these concepts shortly, but in general if a metric does not satisfy those four conditions, it either needs to be replaced, or abandoned.
The first principle is probably the hardest to stick to. It’s hard becasue unless you’re a fool you’re hiring clever developers, ideally cleverer than you. They like solving complex puzzles and playing games. Any metric you create will be gamed. The lazy manager puts rules and policies in place to avoid the obvious gaming. This not only creates an us-and-them dynamic, but it also just makes the metric more complex, more challenging and thus more interesting to game.
A better approach is to anticipate the gaming, embrace it even. But this is not always possible. e.g. Years ago, during a bug-fix crunch period, before we’d adopted Agile, we had a product with a lot of bugs to fix to hit a release readiness milestone. We experimented with some gamification (that’s a topic for another post) but the practical upshot was that we offered prizes for the more days in a row that devs marked bugs as fixed. They only had to fix one bug in the day for it to count.
Now the senior devs could probably fix more than one bug a day, and we were concerned that they’d fix bugs and then sit on them, doling them out each day to get the maximum number in a row. But the thing was, over six weeks, we’d still get 30 bugs out of each of them, so why care.
The second principle is a little easier to stick to. It’s one of the reasons I’ve been harping on about ensuring that the tools you use can be queried and automated. If a metric is generated by someone counting the sticky notes on a wall, human error will always be a factor. Specifically, having a person generate a metric allows the veracity of the metric to be doubted. You can argue with a person and question their ability but you can’t argue with the results of a SQL query.
The third principle is tied closely to the second. Only automated metrics can be produced on demand. Generally if a lot of human processing is required to produce a metric it might be produced weekly or even worse monthly. Imagine how useless a sprint burn-down chart would be if it was only produced at the end of the sprint. That metric is only useful becasue it’s reviewed every day by the team. The moment someone is having trouble it’s obvious and the team pitches in to help.
The fourth principle is probably the most contentious. Metrics influence behavior most strongly when the team can access their metrics on demand and they know that you, and their peers can also access the metric. Think of all the sales teams that have leaderboards, or call centres that show how many calls operators answer. It’s the visibility of the metric that makes this work.
But this is a two edged sword. If the metrics focus down to individuals then you most definitely get developers locally optimising their own numbers even if that’s at the expense of their peers. Software development is a team activity, so in general you only want metrics that are measured for a whole team. Not wanting to let your peers down is probably one of the strongest motivators you have. You can leverage this by measuring the whole team, not individuals.
What are some examples of good and bad metrics that are specific to sprint team management, I hear you ask.
</ul> To sum up, metrics are about communication. They not only tell you what your teams are doing, they tell your teams what you care about. Careful metric choice can also encourage teamwork. When a team’s metrics are published they can’t help but compare themselves to other teams. So metrics need to be chosen with thought becasue like it or not they influence behavior.