The Metrics Monster

Some people are scared to death of metrics. Over the last several decades, attempts to measure throughput or quality have almost always been a harbinger of layoffs, performance improvement plans or taking a significant percentage of the company’s staff overseas. I have been burned on more than one occasion by ISO, CMM, TQM and the other TLAs so I, too, have been conditioned to loathe metrics. They are usually painful to collect, complicated to compile and always seem to fall somewhere along a continuum between Counterproductive and Punitive. But it goes deeper than attempts to measure project ROI or individual value contribution. Our loathing for metrics, regardless of our role in the organization, actually springs from the heart of the daily business in old style Engineering teams.

Although traditional Project Management insists upon the mythical Unchanging Specification, everyone knows that Product Managers can’t think of everything in advance when writing a PRD. Our playing along with the fantasy doesn’t result in eliminated churn but instead forces Product Managers to be creative about how to get changes in without being punished – things like inclusion of vague sections that they can later “clarify” to include forgotten requirements. Of course, Engineering knows Product Management has to do this so they inflate their estimates even more than they already had to account for their own unknowns. Then Management, who has been through this many times before, cuts the delivered estimate by half – effectively insisting that everything go perfectly the first time. Management dictates a short date to a Project Manager, who grabs a Gantt chart and sets out a series of milestones that become millstones nobody wants to support, and the self-delusional project is underway. It’s an unhealthy cycle that is based on an assumption of malice, stupidity and laziness though nothing is usually further from the truth. And it’s an approach that gives a false feeling of control but increases surprises and disappointments even as it tries to eliminate both.

The truth is that most employees genuinely want to contribute value to the organization but their varied skill sets mean that they have to work together to identify where that value is and how best to achieve it. The inherent contention in a traditional Project Management organization prevents that collaboration from even being a point for consideration. Now that’s not to say that traditional Project Management is incapable of delivering. Of course they can, and have for decades. But the organization spends so much time and energy dealing with the nested half-truths and webs of self-preservation that their productivity becomes artificially limited well below what it could be.
We know that there are certain skills that humans naturally possess.

Similarly, there are skills that we do not have. Traditional Project Management is, unfortunately, built around three things that humans are horrible at doing:

  • Creating an exhaustive list that includes all assumptions, risks and caveats. If you want to see how bad you are at this, have a group of several friends maliciously comply with your directions and then give them instructions to walk across the room and turn a light off. Watch the variety of assumptions or details that you forget to state as they collapse into the floor or stumble over one another!
  • Estimating quantity, duration, distance or mass. Think this one is wrong? Next time you go to a new restaurant, estimate how many minutes it will take from the time your server takes your order until the time you have finished eating your meal. Or if you’re really adventurous, walk up to a few strangers on the street, estimate their combined weight and ask them to total it up for you.
  • Refusal to adapt. One of the reasons humans have done such an exceptional job at dominating the Earth is that we adapt so quickly that we don’t even realize we’re doing it in some cases. How did your commute to work go over the last five days? Was each day exactly the same, or did you have to adapt? Our minds simply aren’t wired to follow a set of plans or behaviors while ignoring changing environmental feedback.
  1. Everyone at every level accepts that facts and goals around the project will change.
  2. Working relationships are established to reward and encourage trust, openness and communication.
  3. The whole organization assumes responsibility for dealing with change, not just the person or group who discovered the need for it.

Healthy teamwork requires trust – which is the last thing that traditional Project Management encourages in a team. But by shifting the metrics we collect and how they are used, we can begin to see a healthier and more productive environment begin to emerge. So instead of focusing on “exhaustive lists” where we know we will fail, let’s state the known and agree to discover the unknown together, which is something we all do well. And let’s give up estimates in quantity, duration, distance or mass so we can provide estimates based on complexity and grouping of similar items, at which we excel. And finally, instead of punishing deviation from a plan as the project proceeds, let’s reward adaptation that leads to better, more accurate or satisfying deliverables.

These are three of the basic tenets of Scrum, and why it achieves higher quality, more meaningful results and faster deliverables over time. By playing to the natural strengths and interests of people on the team, everyone’s quality of life is improved, from the Customer to the CEO. With a Scrum team,
And, yes, Scrum does create some metrics but they are not the same old metrics from traditional Project Management. Because Scrum focuses on the known and the possible over the desired, and makes the entire organization responsible for the adaptation to the discovered, these metrics can exist safely without being weaponized.

So when you’re first getting into Scrum, you may find yourself having an alarmed reaction to the metrics it creates. On a team of seven people, usually at least one will say that Scrum feels like micro-management at the beginning. But over time, provided with positive support and reinforcement, most see that Scrum is liberating them from the deceptions and inflations encouraged by the serious problems of traditional Project Management and, by allowing them to increase their value contribution, their job satisfaction begins to soar.

The Myth of Multitasking

I recently had an engineer ask me, "Why does Scrum make me work so slowly? I felt like I was getting a lot more done before we started using Scrum." It's a fairly common question, so I thought I'd post the answer I gave him and a few details that came from our discussion.

His frustration came from the fact that he bought into the myth of multitasking. The more correct name for "multitasking" is Rapid Context Switching (RCS) – a skill that humans don't really have. Nevertheless, he felt like he was accomplishing a lot because at any point in time he would have four or more projects in progress. He felt that it increased his credibility among the engineers if he could provide a long list of projects whenever someone asked him what he's working on, and he would look back at each day with great pride for having been able to move each forward a bit. He was measuring his contribution in lines of code... but there are problems with that approach.

First, all of his projects were always in motion right up until the final deadline. As he crossed the finish line, he would do a Big Reveal to show his team which ones were finished and which ones weren't. He couldn't do it before the last day of the project because he didn't know himself which ones he would finish. His failures to deliver, combined with the surprise factor at the end of the project, would launch the rest of the team into a scramble to account for his missing code. As you can imagine, he wasn't very popular among his peers but Management loved him because they saw him as the guy with the highest bandwidth. He had fooled them into measuring work in progress rather than work completed – which brings me to the second problem with this approach.

As a software engineer, you create value when you have user-relevant code working correctly in production. Until your code is available for revenue generation, you really haven't contributed any bankable value for the company.

Finally, the RCS approach usually increases the time to delivery for any given project. It emerged because we as an industry were too indulgent with un-prioritized pools of work so the scatter shot approach increased the odds that you had done the right thing. But Scrum removes that problem altogether by insisting that the process starts with a triaged Backlog, ordered by Business Value. With that clarity, we no longer have to guess how to maximize our contributions.

We have to change how we think about value contribution and we have to shift our rewards to be in line with this new realization. We have to stop encouraging people to declare success and walk away until the code is in production and working to generate revenue for the company. In short, we have to measure people by what they complete, not what they start.