At a recent performance management workshop, I asked an attendee for an example metric he was using to track performance. The attendee, who worked for a city’s public works department, immediately replied “# of miles of streets cleaned.” Before I could ask any questions, he proudly added “and we cleaned more than 1000 miles last quarter; up 10% since the previous quarter.”
I’m not an expert in street cleaning but 1000 miles and a 10% increase both seemed like good performance improvements. Therein, however, lies the problem with our old friend: context. The attendee (we’ll call him Joe) had attended most of my workshop and was trying to provide additional information about the actual miles of streets cleaned by comparing it to the previous quarter’s actual value. Clearly, an increase of 10% is easier for me – as a non-expert – to understand than the raw number.
However, because this is an activity measure, all I really know is that output increased from last quarter. What if a new subdivision opened in the last 90 days so that the total number of streets that could have been cleaned is 15% higher now? This means that the 10% increase in actual output is further away from our target of cleaning all of the streets. Output is up but performance is down.
Immediately seeing my point, Joe suggested they switch to ‘% of street miles cleaned each month‘ with a target value of 100% and letter grade scoring so that greater than 90% would be an ‘A’, 80-90% a ‘B’, etc. This was a definite improvement and might have ended the discussion until one of the other attendees pointed out that there is an incremental cost to cleaning a higher percentage of streets. Do we really need to set a target of 100% so that every street is cleaned monthly? Is there sufficient value to increase the percentage from 80 to 90? Perhaps the right answer is to set the target slightly higher than the benchmark from our sister communities. That will make sure that we don’t spend too much money but that we continue winning those awards for the best places to live.
That discussion brought us back to one of my favorite topics. The problem with this and other output measures is that it’s not easily tied back to an outcome. What goal are we trying to achieve? After scratching his head for a few minutes, Joe remembered one of the overall objectives for the Public Works department was to increase citizen satisfaction. Presumably, if the streets were clean, people would complain less and they would be happier with the department.
This is logical, but it’s worth going the next step. If the goal is to increase satisfaction, we might establish an outcome measure such as ‘# complaints about street cleanliness reported every quarter‘ with the target of reducing it by 5% every quarter for two years. If we are hitting the target for the output measure of street miles cleaned but not our target for the outcome measure of complaints, we’re probably tracking the wrong activity and unlikely to meet our citizen satisfaction objective. We have a dirty KPI. Perhaps people care about more than just clean streets.
Speaking of which, Joe, there’s a pothole in front of my house I would like to talk to you about.
Nice “concrete” example of KPI’s.
This is a live debate for sustainability performance management also. How do you set a corporate sustainability kpi without understanding the overall sustainability context not just within the firm but beyond it in society and the natural ecosystem? eg. should a firm cut its emissions by 10% or 20%? Sure it depends on capacity of the business to deliver based on its existing business model but it also depends on the context of the environmental limits of the ecosystem beyond the firm and over a longer time horizon.
http://www.globalreporting.org/ReportingFramework/G3Online/DefiningReportContent/LowerBlock/SustainabilityContext.htm