Taking Tests

Sometimes I think that performance management is invading my personal life. A junior high school student that I have been mentoring happily announced during one of our recent meetings that she had gotten a 46 on her latest algebra quiz. Now, I’m no math major but I seem to remember that anything under 60 was a failing grade. How could she be so pleased with her performance?

Not wanting to spoil her celebration, I mulled the puzzle over in private until a possible answer occurred to me. My mentee had reported her actual performance to me without also telling me the target performance. This is one of the problems with business intelligence that performance management tries to solve. Presumably, she had gotten a 46 (actual) out of total possible value of 50 (target). She, and everyone else in the class, knew that information and thus were domain experts. As an outsider, I lacked this context, which lead to my confusion.

Pleased with having figured out this puzzle, I took the next logical step and privately speculated that the teacher could have avoided this communication problem by reporting scores based on percentage attainment, rather than the raw number correct. 92% is much more understandable than 46. Even I would have recognized that as good performance.

Unfortunately, my sense of satisfaction came to abrupt end when my mentee explained that the total possible score was 100. And she was pleased with a 46 out of 100. It took several more minutes of questions but I eventually unearthed the explanation. It turns out that her 46 was the third best performance out of a class of 60 students.  Apparently the test had been too hard so the teacher had decided to grade on a curve. Her 46 was an ‘A’.

Once again, performance management provides language to deal with this situation. Normally we assume that actual performance must be very close to target performance to warrant a high grade. In classrooms, usually 90% and above is an ‘A’. Said another way, the gap between actual and target must be less than 10% to earn the highest grade. In the case of my mentee’s quiz, the gap could be much larger.

Understanding the size of this gap ends up being one of the most important things you can do when communicating performance. If you are a newspaper printer and decide to use defect percentage as one of your measures to track operational efficiency, it might be reasonable to set a target of zero defects but be willing to assign a high grade if 5% of a production run contains printing defects. On the other hand, if you manufacture airplane engines, even a 1% defect rate is probably unacceptable. Two different organizations using the same measure with the same target but with completely different notions of the acceptable size of the gap.

At 15 my mentee isn’t thinking about performance management.  Instead, it’s who her friends are, whether she’ll make the soccer team this year, and what she’s going to wear to the dance on Saturday night. But just yesterday when she was bragging to me about her corner kicking skills she said something like, “My city league team scored on 8 out of 20 or so kicks I took last year. That may not seem like much but most teams would only score 4 or 5 times out of 20.”

I smiled. Target, actual, gap. Performance management is everywhere.

7 Responses to Taking Tests

1. timoelliott July 7, 2015 at 8:32 am #

Isn’t the conclusion rather that in real life, any fixed target (and the gap size) is likely to be arbitrary and irrelevant to measuring performance? What your mentee is saying in both cases is that relative success is all she cares about….

• Jonathan Becher July 7, 2015 at 8:58 am #

To me, this is the essence of true performance management. It does not track activities but rather outcomes — “relative success” in your words. Formally, I recommend three things: actual (measured value at a specific moment in time), target (desired value at the same moment in time), and a grading system (apriori method of judging the difference between the actual and target). In particular, grading systems are not well defined in most situations. More information here: http://jonathanbecher.com/2010/09/27/quick-guide-to-performance-management/

2. Prashant Zaveri January 25, 2016 at 10:22 am #

The better way to communicate performance is to use percentile. Where 99 percentile would be the person performing at top of the class. However, problem becomes much more complex in business environment. Specially when you are ranking team of consultant who have worked on varying projects. A person receiving excellent grading in easy project would come ahead of person receiving good on complex project.

1. - July 20, 2009

[…] Performance management continuously invades my personal life, whether it’s on airplanes, during mentoring, or at Thanksgiving dinner. Here’s another […]

2. - September 27, 2010

[…] this common scheme doesn’t work in many situations such as rewarding over-performance , grading on a curve, and dealing with defect […]

3. - November 24, 2014

[…] I’ve said before, performance management seems to be invading my personal life. Somehow I looked at Thanksgiving […]

4. - August 28, 2020

[…] management continuously invades my personal life, whether it’s on airplanes, during mentoring, or at Thanksgiving dinner. Here’s another […]