Programmed to Comply Leaders Attitudes Toward Surveys

I must admit that I had some fun with a recent Leadership Pulse study. It was October, and perhaps it was the mischievousness of Halloween that led me to devote the study to the topic of surveys. In fact, I used a survey engine to ask questions about how people like taking surveys. And I used a loaded word in two of the questions.

    I asked participants the degree to which they thought the annual employee survey and customer surveys were “evil.” Then I followed up with a few other more traditional questions about surveys.

    I took care to define the word “evil” before I asked survey participants to answer the question. Evil was defined as “a situation that is very unpleasant, harmful or morally wrong.”

    As you might expect, most of the 307 respondents were content enough with surveys to decide that they were not actually evil. (The Leadership Pulse sample consists of leaders who have opted into the study. About 40 percent of the sample are C-level executives, and about 75 percent are director level and above. About 4,000 individuals are part of the core group.)

    The table below reports the percentage of people who agreed (using a scale of 1 to 5, where 1 equals “strongly disagree” and 5 equals “strongly agree”; respondents who agreed answered 4 or 5 on the scale).

   You may wonder why anyone would think a survey is evil. Below are some of the prevalent comments that explain these attitudes:

• “They are poorly worded, do not address the real issues, fail to be acted upon constructively and are typically used to manipulate employees.”

  
  

• “People have learned that surveys can be manipulated, so the importance of surveys has been minimized.”

  
  

• “Auto dealership surveys tell you, ‘They have to have an “excellent” response to all questions.’ This is intentionally skewing the data.”

  
  

• “Most of the time the information goes into a black hole or is used to ‘beat people up’ for not making the right scores. More often the focus is on fixing the numbers instead of understanding what is being said. The last one we did, the CEO did nothing with the information.”

  
  

• “Surveys do not lead to improvements. It seems more like a ‘check the box’ exercise.”

  
  

• “The problem with most surveys is the lack of feedback and action after the data is collected and analyzed. Most leave you with a sense of ‘Why did I bother?’ “

  
  

    Clearly, there are people who are not having positive experiences with taking surveys. However, even with those negative experiences, these individuals were willing to take time to share their opinions and experiences in my survey about surveys.

Other dimensions of surveys
   In addition to the “evil survey” questions, I asked a few more traditional questions about surveys. The overall responses are in the table below:

   As you review these additional scores, it becomes clear that although no one really considers surveys “evil,” respondents did not think very highly of them either. In fact, the comments are in general not so favorable. And as you dig into the data, you quickly see that the most favorable comments come from the people in jobs most likely to do surveys:

    From individuals in marketing:

• “When a company conducts a survey, I feel they are making the necessary steps to want to improve as a business.”

  
  

• “I have participated in my own companies’ surveys and as a customer of another company. I feel providing feedback is critical to improving the customer experience”

  
  

    From individuals in human resources:

• “I have written them, and have participated in them. I believe, if done well, they can be a valuable tool in information gathering and continuous improvement.”

  
  

• “If surveys contain relevant content and are used they work great.”

  
  

• “Surveys are a valuable tool for leadership to assess the attitude of the workforce.”

  
  

    I don’t know what your conclusions are right now, but my take, after reading all the comments, is that even the positive comments are not very positive. In general, people are very ambivalent about surveys. They don’t really “hate” them (although I did not ask that question specifically). But they don’t like them either.

Programmed to comply?
    Could it be that no matter how much we dislike surveys, we are simply programmed to comply with requests to take them? Is it something left over from school, when we weren’t allowed to say no to tests, but simply had to start filling in bubbles and checking boxes?

    There seems to be something to this idea. I meet many companies that do big surveys. Although they sometimes spend millions of dollars on annual employee surveys or customer surveys, I rarely find anyone who has a documented ROI or result from a survey.

    My hypothesis is that there are two competing models at work in the survey world. One is the focus on numbers and scores. There is an entire industry in getting benchmark data. Companies feel a keen need to compare themselves with others, and to do this, they need data.

    Thus, many surveys are really designed just to get a score, and to compare the company to other organizations in their industry. When this is the goal, perhaps there is really no interest in doing anything with the data. So, if this is the case, why not be honest about it? Why not just tell the employees and customers that this is a “score-focused” survey. Answers will be used to provide an assessment of our organization. Period.

    Then there are surveys designed to truly engage people in a dialogue about change in order to drive results. People like these kinds of surveys. Employees and customers want to know that their voice is being heard.

    Traditional, long surveys are fraught with too many problems to truly provide an avenue for voice. If you want to do a survey for change, then the rules need to change. For example, make the surveys short, ask questions that a manager will appreciate (they can take action on the answers),and create a process that starts a conversation that leads to action. In this type of survey, the data are used to start a conversation, not to be an end point or benchmark.

Biased surveys
   Surveys can indeed be biased. In fact, three of the Leadership Pulse participants did a great job of “dinging” me for pursuing a biased survey:

• “I’m concerned with your ‘evil’ bias. I would have expected a more objective mind-set.”

  
  

• “I have a problem with the way some of these questions are written: biased, leading, extreme.”

  
  

• “The use of the word ‘evil’ in this context seems quite out of place and overdone.”

  
  

    But every survey is biased. Whenever you choose to ask a certain set of questions, you are biasing the respondents to think about what you want them to consider. That was my goal. I am simply tired of the confusion about surveys, and I wanted to seek out some additional opinions on the topic. I also was ready to have some fun with the topic. Humor engages people in the conversation a bit more than a boring, academic survey can.

Data that open a dialogue
    In the first page of the “evil” survey I was quite open about my biases. I told the respondents that I was in search of an alternative for surveys. I’m not interested in creating the “perfect survey,” because I believe there is no such thing. No magic questions. No perfect constructs.

    I am convinced, however, that you can use survey data to initiate incredibly rich conversations. Data and dialogue about the data are the magic. You can entice people to discuss topics that were taboo. I have done this with the most senior of executive teams. The data are “magic” because it allows everyone to share an opinion about the data versus an opinion about themselves. Consider how well this can work with mergers, for new leaders, or in organizations going through dramatically high rates of change.

    But data for dialogue will be very customized. It will not provide good benchmark data. The key to keeping the surveys “not evil” is that, regardless of goal, the people running the surveys admit how they are using the data. If purpose is honestly communicated to survey participants, there will be fewer complaints and higher participation, and the result will be improved relationships with survey participants.

If you want to join the Leadership Pulse study and receive technical reports on these data, you can sign up here.

Click here to view the complete technical report of these findings and other leadership pulse reports.