Dangers of Open-Ended Questions to Participants in 360-Degree Feedback

July 13, 2016 by Sandra Mashihi

“The other day I got out my can-opener and was opening a can of worms when I thought, What am I doing?!”

-Jack Handey

The usefulness of having qualitative comments in 360-degree feedback interventions is not widely debated. In fact, before online administration of 360-degree feedback assessments was even possible, most coaches and consultants relied on summarizing interviews with key stakeholders to share feedback themes with participants. Typically in either vendor developed 360 assessments or customized ones designed in-house, open-ended questions are typically included voluntarily and confidentially answered by raters.

Of course, narrative comments can possibly be evaluative, overly critical, or negative, having an adverse impact on acceptability. For example, Smither and Walker (2004) analyzed the impact of upward feedback ratings, as well as narrative comments, over a one-year period for 176 managers. They found that those who received a small number of unfavorable, behaviorally-based comments improved more than other managers, but those who received a large number relative to positive comments significantly declined in performance more than other managers. Like quantitative results, open-ended comments can create strong emotional reactions that can interfere with the acceptance of feedback and lead to diminished engagement and productivity.

In most 360-degree feedback interventions, invited raters are often not given any type of training of guidance to maximize the usefulness of their written comments. Ideally, invited raters should be communicated with early on to describe the purpose of the 360-degree feedback intervention, how their ratings will be used, who will have access to them, and how to make them as behavioral, constructive, and useful for the participant as they can possibly be. This type of communication and training can be helpful to reduce both the leniency and strictness errors and other forms of rating biases.

Coach’s Critique:

One would think that qualitative data would be more beneficial than quantitative data. However, there are pros and significant cons to the outcome of qualitative data. Receiving too much information can have some major pitfalls!

Since questions on the quantitative portion of the 360 are posed in a positive way, participants vulnerability to negative feedback is minimized. As it is, with the quantitative part of the 360, participants often feel overwhelmed by the feedback despite the fact that data is posed in a controlled and somewhat of a positive way (e.g. based on strengths and improvement opportunities). Rather, qualitative data can “open up a can of worms”. Feedback can be negative and harsh. As a result, participant’s esteem to make behavioral changes can actually be shattered!

On the other hand, there may be specific feedback suggestions within the qualitative data that is not covered in the quantitative portion of the 360. Therefore, eliminating qualitative data probably wouldn’t be the best solution. So, if you are a coach or consultant, you play an important role in moderating the effects of the feedback. Here is a suggestion to keep in mind…

You may want to emphasize the themes surrounding the qualitative comments, rather than, focusing on a single comment that might represent a single individual’s experience, perception, or reaction.

What have been your experiences with the effects of qualitative feedback in 360s? How have you balanced between quantitative and qualitative feedback? Any suggestions?

Dr. Sandra Mashihi is a senior consultant with Envisia Learning, Inc. She has extensive experience in sales training, behavioral assessments and executive coaching. Prior to working at Envisia Learning, Inc., She was an internal Organizational Development Consultant at Marcus & Millichap where she was responsible for initiatives within training & development and recruiting.. Sandra received her Bachelor’s of Science in Psychology from University of California, Los Angeles and received her Master of Science and Doctorate in Organizational Psychology from the California School of Professional Psychology.

Posted in 360 Degree Feedback

If You Enjoyed This Post...

You'll love getting updates when we post new articles on leadership development, 360 degree feedback and behavior change. Enter your email below to get a free copy of our book and get notified of new posts:

  • We use a 360 degree instrument that is not anonymous as pre-work for our Exercising Influence program. There is an opportunity for respondents to provide comments. Because we have framed it as a way for individuals to provide suggestions as to how the subject could be more influential with them, the comments are generally perceived as valuable insights rather than as critical feedback.

Follow Envisia Learning:

RSS Twitter linkedin Facebook

Are You Implementing a Leadership Development Program?

Call us to discuss how we can help you get more out of your leadership development program:

(800) 335-0779, x1