Based on the statistics provided to me by WordPress, one of the most popular entries I’ve written was Dirty KPIs. In it, I talked about the challenges of using an activity metric like “# of miles of streets cleaned”. Based on a workshop attendee’s assertion that one of his organization’s objectives was to “increase citizen satisfaction”, I suggested that an outcome measure such as “# complaints about street cleanliness reported every quarter” would be a better fit. My theory was that if the organization is reaching its target for the output measure of street miles cleaned but not its target for the outcome measure of complaints, then it’s probably tracking the wrong activity and unlikely to meet its citizen satisfaction objective.
Not long after I wrote that blog entry, a column entitled “San Jose street sweepers miss mark” appeared in a local newspaper. In it, three separate readers complain about the local street sweepers. If the City of San Jose had been using my suggested complaint metric, their score for street cleaning probably wouldn’t have been very high.
Even more telling, however, is that two of the readers assert that the street cleaners don’t actually clean the streets. This shows another flaw in using output metrics like “# of miles of streets cleaned”. Sure, the machine passed over many miles of streets but if citizen don’t believe that the result is cleaner streets, did the organization achieve any outcome at all? Did it have any impact? Probably not.
This example also ties back to another popular blog entry about point of view. Normally, we think about street sweeping from an internal process point of view. The number of miles of streets that were cleaned represents a measure of our own activity. However, citizens – representing the customer point of view – don’t care about how hard we worked. They want their streets to actually be free of debris.
No matter how efficient our processes are, if the customer isn’t satisfied with the result, it wasn’t very effective. And we should get a poor grade on performance.