After many years as a performance management enthusiast, I sometimes forget how much confusion there still is around the topic. Since I’m a big believer that standardized language helps reduce confusion, I’ve decided to summarize some of my deeply held beliefs on performance management:

An objective describes what you want to accomplish. For example, ‘win the race’. A key performance indicator (KPI) monitors progress towards a specific objective. For example, ‘position among the runners’. A target is the value of a KPI at a defined moment in time. For example, in order to win the race, the target value for the KPI needs to be 1 at the end of the race (but not necessarily throughout the race).

Obviously the actual value of the position KPI can change over time, as various runners speed up and slow down (or even leave the race). In a similar manner, you should apriori set target values for different moments of time. That way, you can compare your actual value to the target value at specific intervals to know if you are on track.

(This 2008 post describes more about objectives, KPIs, and targets using the analogy of a race.)

I’ve noticed that many people forget that targets can – and should – have multiple values and instead focus on the ultimate result: zero defects, $100M in revenue, or a 4.5 score in customer satisfaction. If the result is far enough away in time, comparing the current actual value to the eventual target value may lead to the mistaken conclusion that we’re not going to reach the objective. This is especially critical when results don’t happen uniformly. For example, in high tech sales, a disproportionate amount of deals happen in the last month of a quarter and in the last quarter of a year. A single target value would not capture this nuance.

(This 2007 post provides an example of multiple targets in a call center.)

A grading system compares actual and target values to determine a performance score. The closer the actual value is to the target value, the better the score. One of the most common systems is based on letter grades (A/B/C/D/F) in which 90% and above is an “A”, 80-90% is an “B”, etc. Unfortunately, this common scheme doesn’t work in many situations such as rewarding over-performance , grading on a curve, and dealing with defect rates.

In my experience most performance management practitioners don’t spend enough time creating the appropriate grading system that will help them understand (and explain) the size of the gap between actual and target. Unfortunately, this oversight means many of their performance conclusions are faulty.

Finally, organizations may use measures to track the resources that they consume and the product/services they produce but should focus KPIs on the results that they are trying to achieve. This distinction is codified in the Logic Model which describes the relationship between inputs, outputs, and outcomes.

The distinction between output and outcome also gives rise to some of my favorite performance management stories: the dirty KPIs in a city public works department, circumventing the CRM system in a call center, and keeping my own lawn green.

Do you have your own story of outcomes vs outputs that you’d like to share?

The post Quick Guide to Performance Management appeared first on Manage By Walking Around.

For those of you who remember the series of posts two years ago defining objectives for this blog, you’ll recall that I decided that average number of comments per post and number of unique visitors per month were better measures of my success than number of page views or frequency of posts.  Regardless, I was more than a little disappointed when I realized that I’ve gotten out of my weekly rhythm this summer.  As I haven’t had any time to work on my personal writing in almost two years,  I’ve missed more than 5 posts this summer; I’ve missed a creative outlet.

Given that thought process, I wonder if frequency of posts might be a reasonable metric for me. Curious what others might think, I did a little informal research.  Not surprisingly, there are dozens of posts about how to measure the success of the blogs.  Most of the insightful ones seem to focus on the number of inbound links (essentially citations) or the average number of comments per post (essentially conversations).

However, my favorite summary came from a post entitled Blog Metrics: Six Recommendations For Measuring Your Success.  Avinash recommends six categories of metrics:

1. Raw Author Contribution 
2. Holistic Audience Growth
3. Conversation Rate 
4. Citations/ Ripple Index
5. Cost
6. Benefit / ROI

While Avinash’s measures of raw author contribution don’t include frequency, he does include this note: 

The important thing is to measure: consistency. I find it builds traffic.

I agree, so frequency of posts is back as a measure to watch. And since Sunday posts seem to drive the most traffic, I know how I’ll be closing out my weekends.

The post Blog Metrics: Frequency of Posts appeared first on Manage By Walking Around.

Performance management continuously invades my personal life, whether it’s on airplanes, during mentoring, or at Thanksgiving dinner. Here’s another example:

Not long after I moved into my previous house, a friendly neighbor came over to welcome me to the block. Amid advice on local stores and restaurants, he pointed out that my front lawn wasn’t up to the neighborhood standards. In fact, it was mostly brown and barren. I needed green grass.

Wanting to keep up with the Joneses, I hired a landscaper who produced incredibly detailed designs, brought in a wide variety of equipment, and kept my front yard in shambles for weeks. When he was done, I had lawn, hedges, flowers and a bill much higher than I had expected. But I had a yard to make the neighbors proud.

Unfortunately, this extreme makeover took place in the summer heat of California and, despite frequent watering, within two months my front yard was well on the way to becoming brown and barren again. I summoned the landscaper who discovered some of the sprinklers had very little water pressure. He proceeded to blame 1” PVC tubing, long runs of pipe with sediment buildup, and an aggressive ground hog. I didn’t understand or care. I just wanted green grass.

The landscaper tore up my lawn again, replaced some pipes, and installed a trap. Water pressure was restored but the lawn didn’t recover. Another return trip, another diagnostic, and another detailed explanation — this time of the acidic clay in my soil. Apparently my yard wasn’t the ideal environment for fescue grass. I still didn’t care.

Frustrated and with more money out of pocket than I care to admit, I called a yard specialist recommended by a co-worker. While I don’t remember the exact conversation, my interview with Mr. Yamagushi went something like this:

Me: Can you install a lawn that will last longer than 6 months?
Yamagushi: Why?
Me: Because the one I have keeps dying.
Yamagushi: No, why do you want a lawn?
Me: Because my neighbors are embarrassed by my front yard.
Yamagushi: OK, I’ll give you a non-embarrassing yard.

At which point he wrote up an estimate that said “Yard to be proud of” with a price much lower than the original landscaper. I tried to get him to explain what that meant but he kept saying that he would worry about the details. If I wasn’t happy, I didn’t have to pay.

The next day I got home from work to find my lawn ripped out, replaced by a Zen rock garden, some drought resistant plants, and a few areas filled with bark mulch. It was beautiful. And it lasted with very little maintenance for the remaining years I lived in that house.

So what does this story have to do with performance management? The original landscaper peppered his stakeholder (me) with activity metrics that described largely meaningless activities. Mr. Yamagushi instead emphasized an outcome KPI with an easily articulated impact (lack of embarrassment). Yard outcome beats lawn activities.

We should all examine the dashboards and scorecards we publish to our stakeholders. Are they filled with activity metrics that showcase our hard work but that are potentially uninteresting to our stakeholders? Or do they contain a small number of outcome KPIs that clearly show impact?

Let’s prioritize outcomes over activities.

The post Lawn Activities, Yard Outcomes appeared first on Manage By Walking Around.

One part writer’s block and two parts schedule overload means I don’t have time to write an original post.  Speedlinking to the rescue:

Over at ‘I Help You Blog’, Philip suggests 101 Great Posting Ideas That Will Make Your Blog Sizzle.  My personal favorite is #101:  Create a post with a 101 ideas.

In a relatively new blog, Michael and Barry write about many of the same topics that I’m interested in, including key risk indicators, setting targets, and the problems caused by complexity.   I’ll be reading them regularly.

In Leadership: Making the Tough Decisions, Don Bulmer provides three basic steps to making tough decisions.  Timo Elliott suggests how technology should help make decision making easier in the future.  Technology can definitely help but I’m not sure it will ever replace intuition and experience.

The BiPM Institute believes that people become the way you treat them and recommends that every manager ask his/her employees what they would do if they were in the manager’s place.   I think managers would be taken aback to hear what their employees would suggest.

And finally, Adrian complains that only 15% of UK government councils make information easier to find and consume by using RSS.   Whether in public sector or for-profit enterprises, strategy is too often a secret from core stakeholders (citizens and employees).   Strategy, like information, wants to be free.

The post Speedlinking, May 2009 appeared first on Manage By Walking Around.

As I’m done bashing MBOs (for now), I might as well revisit my long-standing concern with the ubiquitous red/yellow/green traffic light metaphor. While the metaphor is intended as a simple summary of performance (green = good, red = not good), for most business situations three levels of performance are not enough to truly judge results. A simple – but compelling – example of this disconnect came from a recent exercise in helping a sales team design their MBO targets.

This team designed a dashboard that showed each region’s and individual contributor’s achievement versus sales quota (ie. the target). They recognized performance should be compared against quarterly targets, not just yearly ones. However, because quotas varied widely by individual and region, sales management didn’t want to display actual or target values but rather an “at a glance progress towards goals.”

When I arrived on the scene, the team was struggling with how to systematically convert these values into colors without relying on manual intervention. Using my terminology, they needed a grading system.

The team had used a simple device that leveraged the fact that the business plan was based on an average sales rep achieving 85% of their quota: 100%+ achievement was green, 85%+ was yellow and below 85% was red. However, sales organizations typically have an incentive or accelerator that is invoked for achievement at a threshold (such as 120%) above quota. This grading system failed to distinguish between these overachievers and those who met their quota.

After a brief discussion, we concluded that this scheme also masked an issue with low performers. In this organization, some fraction of the sales reps achieve less than 50% of their target for several quarters in a row and are put on a sales improvement plan. Failure to fix their performance can lead to termination. Grading these low performers the same as those who achieved 84% of their quota (nearly the expected performance) was both unfair and inaccurate.

Given the predominance of the traffic light metaphor, the team continued to struggle to come up with a three color grading system until I reminded them that a traffic light is only designed to cover two outcomes: proceed or stop. The third color – yellow – signifies a transition from green to red but doesn’t create a new potential outcome. More on this here.

In our example, we wanted at least three separate outcomes (performance improvement plan, track earned commissions, engage accelerators) and possibly a fourth (notify achievement of quota). Clearly, we needed more than three states and more than three colors. We explored a variety of metaphors (Harvey balls, emoticons, thermometers) but the organization wanted something close to the classic red/yellow/green traffic light. In the end, we decided to keep the colors but get rid of the stacked lights, as in this design: 

The corner cases are clearly handled: over-achievement is dark green while improvement plans are dark red. Furthermore, those “close to” the expected average are graded yellow, even though they haven’t reached it yet. This scheme is simple enough to explain to an individual contributor but provides enough granularity to judge performance.

What do you think?

The post Traffic Lights redux appeared first on Manage By Walking Around.

On Friday, a colleague in another division asked me “How do you measure the effectiveness of analysts relations?” It’s an unanswerable question we’ve all been asked 1000 times about a wide variety of topics (even a marine terminal gate). As I said then, every time someone asks you to show them the measures, be sure to quote Stacey Barr:

These are not measurement questions. These are questions about strategy or direction. If you know what results your work should produce, then the measurement is almost obvious.

After a brief discussion, my colleague determined that he really wanted to measure the efficiency of his analyst relations team so as to benchmark them with other organizations’ size and budget. We agreed that it would be better to use operational metrics rather than strategic KPIs.

This weekend it occurred to me I never really answered the original question. What are the strategic KPIs for analyst relations?  Rather than trying to come up with the answer myself, I did a bit of research and found SageCircle – a firm that positions itself as “analysis of the analysts”.  They have this promising description:

When building AR measurement programs, AR practitioners must distinguish between performance and operational metrics. Performance metrics help AR teams measure progress against strategic goals while operational metrics measure utilization and productivity against plan.

However, their specific examples of performance metrics (i.e. KPIs) are not as illuminating: “analyst opinions by market, product, etc. or number of sales opportunities supported by the AR team.” These still seem like activity metrics, not outcomes.

I found a more appealing idea in an article by Katie Paine entitled “How to Measure Relations with Opinion Leaders in the Media“. In a section on successful analyst relations, she says:

As with journalists, the ultimate measure of a successful relationship is if they recommend your product to reporters, editors and customers.

The idea is to apply the customer loyalty concept of Net Promoter Score to analysts. It measures the desired outcome, has a well-defined grading system, and is easily benchmarked with other organizations. This makes sense to me.

What do you think?

The post Measuring Analyst Relations appeared first on Manage By Walking Around.

For many years, I’ve argued that performance management is not just limited to finance but instead has many flavors, including workforce, operational, and IT performance management. Most people now seem to agree and there’s even talk of pervasive performance management.  However, a critical analysis of the current situation suggests that marketing organizations are still early on in their adoption cycle of performance management.  As a recent Gartner report commented,

Marketing may be one of the largest budgetary items in most organizations, but it is still viewed as a cost center. Developing a marketing performance management framework can improve measurability, make marketing more accountable and align its activities with corporate objectives.

I’m not talking about marketing metrics – there are plenty of those – but rather explicit definitions of strategy and objectives.  Nowhere is the disconnect between metrics and strategy more obvious than for the Web site.  We are awash with solutions that measure Web site performance but very few that connect these measurements to Web strategy.  We need Web channel performance management.

Admittedly, this isn’t a new idea for me. In my previous job, we built a product that applied the Balanced Scorecard methodology to Web site strategy. Rather than exclusively focus on Web site traffic metrics, we provided libraries of objectives for typical Web channel business models, including brand/awareness building, lead generation, and customer support.  This is what a typical strategy map might look like:

It was a good idea and a solid product but ahead of its time. Despite favorable reviews, the product never really took off.

Fast forward four years.  Imagine my surprise when I discovered that someone calling him/herself Webscorecard had started a blog that describes how to apply the Balanced Scorecard methodology to Web site strategy.  One of the early posts provides the following advice on how to build a Web scorecard:

1. Define your Mission and Vision
2. Define your business strategy for the internet
3. Define your limits (your values, what you think is important)
4. Define your stakeholder goals (what is the ultimate direct measure for success?)
5. Define you view on what your customers want from you on the internet
6. Define the processes you need to have do deliver to these customers
7. Define what you need for continued success in the future.

It’s good advice: focus on strategy before metrics.  When you do get around to monitoring your Web objectives, it might look something like this:

Webscorecard: These are the visuals I promised you.  Good luck pursuing Web channel performance management.  I hope you’re more successful than I was.

The post Web channel performance management appeared first on Manage By Walking Around.

An article in Industrial Market Trends alerted me to the Goldense Group 2008 Product Development Metrics Survey of the top 10 R&D metrics used by industry:

• R&D spending as a percentage of sales (77 percent);
• Total patents filed/pending/awarded/rejected (61 percent);
• Total R&D headcount (59 percent)
• Current-year percentage sales due to new products released in the past six years (56 percent);
• Number of new products released (53 percent);
• Number of products/projects in active development (47 percent);
• Percentage resources/investment dedicated to new product development (41 percent);
• Number of products in defined/planning/estimation stages (35 percent);
• Average project return on investment or average projects payback (31 percent); and
• Percentage increase/decrease in R&D headcount (31 percent).

I’m not surprised, but nonetheless a little disappointed, that the vast majority of these are activity metrics as opposed to outcome metrics. The two exceptions are ‘current-year percentage sales due to new products released in the past six years’ and ‘average project return on investment or average projects payback’. As such, they are potential candidates to elevate from operational metrics on a dashboard to strategic KPIs on a scorecard.

Both of these metrics are lagging, rather than leading, indicators of innovation which means that they do a good job of tracking innovation after it happens but may not give us much insight into future innovation. If you were in charge of creating a leading KPI for innovation, what would you choose?

The post Measuring innovation appeared first on Manage By Walking Around.

While most people don’t distinguish between goals and objectives in every day conversation, I find that performance management practitioners sometimes use the word goal when they really mean target. This potential confusion between objectives and targets explains why I usually recommend against using the term goal when establishing a standard performance management terminology.

From my point of view, objectives, KPIs, and targets are related, but distinct, items:

• An objective describes what you want to accomplish.
• A key performance indicator (KPI) monitors progress towards a specific objective.
• A target is the value of a KPI at a defined moment in time.

As always, a simple example is useful.  Imagine a runner who has entered a marathon with the objective to win the race. On the surface, accomplishing that objective is binary – the runner either does or doesn’t win the race.  However, to better understand the runner’s performance, we can create the KPI ‘position at the end of the race’ and set the target to be 1. We judge the runner’s performance by creating a grading system that compares the actual value to the target:

1 = success
2-20 = pretty good
>20 = not acceptable

In other words, anything but a top 20 finish would be viewed as poor performance. We could even map these categories to the classic green/yellow/red stoplight metaphor.

The system is very adaptable. As this runner improves over time, we can change the grading system for future races so that only a top 10 finish would be considered good performance. Another, less accomplished, runner could share the same objective and KPI but create a different target and grading system as follows:

< 25 = success
25-100 = pretty good
>100 = not acceptable

If we’re not just interested in performance at the end of the race but want to monitor progress during the race, we can change the KPI from ‘position at the end of the race’ to just ‘position’. In performance management terminology, we have switched from a lagging indicator to a leading indicator.

If you’re a runner, you may have recognized a potential complication. A runner who is a strong finisher may not expect to be in the top few positions during the first half of the race but rather wait until the last few miles for a closing kick. In fact, any particular race may have a disproportionate number of fast starters (so-called rabbits) that don’t pose a serious threat.

To handle this situation, we can introduce a second KPI that monitors performance for the same objective: ‘average time per mile’. Average time per mile can be measured at various points during the race to benchmark performance. Depending on your skill level, the multiple targets could be 5:00 at mile 4, 5:35 at mile 10, and 5:20 at mile 20. An appropriate grading system might be < 2 seconds difference = success, 2-5 secs difference = pretty good, >5 secs difference = not acceptable. Average time per mile also allows you to compare performance between races and between runners with similar capabilities.

We all want to win the race. But we can do a better job of improving performance if we focus on targets to allow us to reach our objectives.

The post Target vs. Goal appeared first on Manage By Walking Around.

Over at Slate, Zachary Meisel and Jesse Pines have a sensationalist article entitled Waiting Doom that describes how hospitals are killing emergency room patients. They claim Esmin Green’s death was caused by the length of time she waited in the E.R. which they blame on the hospital practice of boarding inpatients. In their words,

Despite increasing evidence that crowded E.R.s can be hazardous to your health, hospitals have incentives to keep their E.R. patients waiting. As a result, there has been an explosion in E.R. wait times over the past few years, even for those who are the sickest.  […]  If there are no inpatient beds in the hospital (or no extra inpatient nurses on duty that day) then the patient stays in the E.R. long past the completion of the initial emergency work. This is what happened to Green, and it has become widespread and common.

To be clear, this death is tragic and, from what I’ve seen and read, there does appear to be some hospital negligence. However, I’m worried that Meisel and Pines’ suggested cure is worse than the disease. Like many who mistakenly believe in activity measures, they recommend we force hospitals to measure and report waiting time. As evidence that this is the right solution, they cite:

In England, the National Health System now has a rule that 98 percent of patients have to spend less than four hours in the E.R.

However, as I blogged in Unhealthy Measures, when the National Heath System mandated a waiting time limit, ambulances waited outside hospitals with patients in the back. Since the activity measure started when patients came in the door, it seemingly solved the waiting problem but didn’t address the real problem – getting patients healthy.  Outcome measures such as ‘% cases diagnosed correctly’ and ‘mortality rate’ would assess the effectiveness of the care being provided, not just the speed at which it was provided.

It’s hard to know whether using performance management would have helped Esmin Green. But I’m certain spending a lot of money to measure waiting time isn’t the answer.

The post More Gloom Than Doom appeared first on Manage By Walking Around.