The Metrics Monster

Some people are scared to death of metrics. Over the last several decades, attempts to measure throughput or quality have almost always been a harbinger of layoffs, performance improvement plans or taking a significant percentage of the company’s staff overseas. I have been burned on more than one occasion by ISO, CMM, TQM and the other TLAs so I, too, have been conditioned to loathe metrics. They are usually painful to collect, complicated to compile and always seem to fall somewhere along a continuum between Counterproductive and Punitive. But it goes deeper than attempts to measure project ROI or individual value contribution. Our loathing for metrics, regardless of our role in the organization, actually springs from the heart of the daily business in old style Engineering teams.

Although traditional Project Management insists upon the mythical Unchanging Specification, everyone knows that Product Managers can’t think of everything in advance when writing a PRD. Our playing along with the fantasy doesn’t result in eliminated churn but instead forces Product Managers to be creative about how to get changes in without being punished – things like inclusion of vague sections that they can later “clarify” to include forgotten requirements. Of course, Engineering knows Product Management has to do this so they inflate their estimates even more than they already had to account for their own unknowns. Then Management, who has been through this many times before, cuts the delivered estimate by half – effectively insisting that everything go perfectly the first time. Management dictates a short date to a Project Manager, who grabs a Gantt chart and sets out a series of milestones that become millstones nobody wants to support, and the self-delusional project is underway. It’s an unhealthy cycle that is based on an assumption of malice, stupidity and laziness though nothing is usually further from the truth. And it’s an approach that gives a false feeling of control but increases surprises and disappointments even as it tries to eliminate both.

The truth is that most employees genuinely want to contribute value to the organization but their varied skill sets mean that they have to work together to identify where that value is and how best to achieve it. The inherent contention in a traditional Project Management organization prevents that collaboration from even being a point for consideration. Now that’s not to say that traditional Project Management is incapable of delivering. Of course they can, and have for decades. But the organization spends so much time and energy dealing with the nested half-truths and webs of self-preservation that their productivity becomes artificially limited well below what it could be.
We know that there are certain skills that humans naturally possess.

Similarly, there are skills that we do not have. Traditional Project Management is, unfortunately, built around three things that humans are horrible at doing:

  • Creating an exhaustive list that includes all assumptions, risks and caveats. If you want to see how bad you are at this, have a group of several friends maliciously comply with your directions and then give them instructions to walk across the room and turn a light off. Watch the variety of assumptions or details that you forget to state as they collapse into the floor or stumble over one another!
  • Estimating quantity, duration, distance or mass. Think this one is wrong? Next time you go to a new restaurant, estimate how many minutes it will take from the time your server takes your order until the time you have finished eating your meal. Or if you’re really adventurous, walk up to a few strangers on the street, estimate their combined weight and ask them to total it up for you.
  • Refusal to adapt. One of the reasons humans have done such an exceptional job at dominating the Earth is that we adapt so quickly that we don’t even realize we’re doing it in some cases. How did your commute to work go over the last five days? Was each day exactly the same, or did you have to adapt? Our minds simply aren’t wired to follow a set of plans or behaviors while ignoring changing environmental feedback.
  1. Everyone at every level accepts that facts and goals around the project will change.
  2. Working relationships are established to reward and encourage trust, openness and communication.
  3. The whole organization assumes responsibility for dealing with change, not just the person or group who discovered the need for it.

Healthy teamwork requires trust – which is the last thing that traditional Project Management encourages in a team. But by shifting the metrics we collect and how they are used, we can begin to see a healthier and more productive environment begin to emerge. So instead of focusing on “exhaustive lists” where we know we will fail, let’s state the known and agree to discover the unknown together, which is something we all do well. And let’s give up estimates in quantity, duration, distance or mass so we can provide estimates based on complexity and grouping of similar items, at which we excel. And finally, instead of punishing deviation from a plan as the project proceeds, let’s reward adaptation that leads to better, more accurate or satisfying deliverables.

These are three of the basic tenets of Scrum, and why it achieves higher quality, more meaningful results and faster deliverables over time. By playing to the natural strengths and interests of people on the team, everyone’s quality of life is improved, from the Customer to the CEO. With a Scrum team,
And, yes, Scrum does create some metrics but they are not the same old metrics from traditional Project Management. Because Scrum focuses on the known and the possible over the desired, and makes the entire organization responsible for the adaptation to the discovered, these metrics can exist safely without being weaponized.

So when you’re first getting into Scrum, you may find yourself having an alarmed reaction to the metrics it creates. On a team of seven people, usually at least one will say that Scrum feels like micro-management at the beginning. But over time, provided with positive support and reinforcement, most see that Scrum is liberating them from the deceptions and inflations encouraged by the serious problems of traditional Project Management and, by allowing them to increase their value contribution, their job satisfaction begins to soar.