Surveying Satisfaction in Software

Early in my management career, the HR department sent out a single-question survey which asked how satisfied we were with our jobs on a scale of 0 (very unsatisfied) to 4 (highly satisfied).  To my surprise, my group had the lowest average score (2.6) in the company. The HR VP predicted a mass exodus and told me that my annual bonus would be negatively impacted. I was mystified because I had heard very few complaints and nobody had left in the previous year.

Not one to leave such mysteries unsolved, I convinced HR to ask my group three more questions:

  • How likely are you to leave the company in the next 6 months?
  • How does your job compare to other jobs that you’ve had?
  • How does your job satisfaction compare with how it was 3 months ago?

The results made me feel better. My group reported they were unlikely to leave the company in the next six months, their current job was slightly better than ones they’d had in the past and they felt the same about their job today as they did 3 months ago. The evidence convinced HR I didn’t have a major problem brewing. And saved my bonus.

Despite this, I was curious why my group’s results were lower than the company average. During my regular one-on-ones in the subsequent weeks, I described the situation to my employees and asked for their opinion.

Apparently software engineers are tough graders. Many said they would never give a higher score than 3.5, no matter how happy they were. It irritated them that the company soda machines didn’t carry Pepsi products; enough to lower their grades.  A 2.6 score for my group might have been equivalent to 3.0 or higher for another group.

This confusion could have been avoided if HR hadn’t chosen to benchmark groups against each other.  Comparing software engineers to salespeople is probably not a good idea. Instead, HR should have ignored the raw results and focused on the trend over time.  Are the results from this quarter’s survey result going up or down as compared to last quarter? Since we only had one data point, I was trying to approximate this by asking, “How does your satisfaction with your job compare with how it was 3 months ago?”

As it turns out, no one left my group over the next year despite the VP’s warning.  While interesting work or a soft job market might have been the primary reasons, I like to think that it had something to do with the fact that I stocked my office mini-fridge with free Pepsi.

(An earlier version of this blog appeared in 2007.)

, , , , , ,

7 Responses to Surveying Satisfaction in Software

  1. Zoy Chiang (@ZoyChiang) May 26, 2013 at 7:16 pm #

    Hello Jonathan,

    Thank you for sharing the strategic decision power of Pepsi. 🙂
    Your article reminds me of my previous VP, he often asked me to score the happy feeling of working with him, from 1 to 10.

    And the question of ” “How does your satisfaction with your job compare with how it was 3 months ago?” did inspire me to check with my direct reports, to further understand if we are providing a growing environment for people development .

  2. Jens Hartmann (@marginalien) May 27, 2013 at 7:45 am #

    confusion could have also been avoided with regard to the methodology they applied…. asking single-item questions always gets you a lot of noise in your data. often, it remains unclear what is actually measured. job satisfaction certainly has many different dimensions. seriously: management by single-item questions isn’t management by figures, it’s really management by noise. survey methodology – validity and reliability of scales and items – is crucial.

  3. Christopher Powell May 27, 2013 at 1:38 pm #

    Agree with trending over time. Also – as you point out, many factors impact the results – sometimes as unpredictable as the choice of beverages in the cafeteria. While there are some instances where a mass exodus should impact a performance bonus, as noted by your HR VP, there are plenty of situations outside of an indivual managers ability to address. It is right to use these kinds of surveys to direct positive change, it is dangerous to use them to assume poor management.

    Is buying Pepsi pandering to your employees?

  4. David Swanson May 27, 2013 at 11:21 pm #

    As one of those “HR people” wanted to comment on this excellent thought provoking piece by Jonathan. As Markus Buckingham of Gallop and Do You Best At Work fame often said, “people don’t leave companies, they leave managers.” I would venture that it was Jonathan’s leadership that kept them around.

    Surveys are a point in time event. There is some reallly interesting work being done on discrtionary effort as the key element of people in successful companies not job satisfaction or even employee engagement.

    People who do “above and beyond” effort are doing so not because it is expected or even for career advancement reasons. They do it because they are fully into what they are doing. Part of it is that they have a feeling of self accomplishment, part is that they are working on topics that are both of interest to them and to the businesses they work at.

    Mostly though as leaders it is our responsibility to help our people know what they are good at, what they have a passion for, and what needs to be done in the workplace. If we can enable that then get out of their way, as I have seen Jonathan do, then they will top the discretionary effort scales.

  5. Carolyn Brock May 28, 2013 at 6:54 am #

    I can relate to the Pepsi detail! When I was managing developers I learned how powerful small gestures can be. We all worked remotely, but in the same city, so I arranged for group lunches once each month, I learned so much during those lunches; they were relaxed enough to share what issues they were facing and to ask for my support in removing barriers. Lunch isn’t always a viable option, so another idea is to copy employees on occasional emails to your peers and managers, commending them on a job well done. Little gestures go a long way in building and maintaining morale and loyalty.

  6. Malcolm Ryder June 1, 2013 at 8:01 am #

    A hugely timely retrospection, Jonathan! With a double-wave of new-generation workers coming into the workforce between now, 2016 (the last crest of today’s education/training styles) , and 2019 (the first crest of the new education/training styles), there could hardly be anything more challenging than to begin showing the connections between their way of assessing things, their way of positioning themselves for manager expectations, and their way of identifying whether they are allowed to do their best work as a result.

Leave a Reply