“The other day I got out my can-opener and was opening a can of worms when I thought, What am I doing?!”
-Jack Handey
The usefulness of having qualitative comments in 360-degree feedback interventions is not widely debated. In fact, before online administration of 360-degree feedback assessments was even possible, most coaches and consultants relied on summarizing interviews with key stakeholders to share feedback themes with participants. Typically in either vendor developed 360 assessments or customized ones designed in-house, open-ended questions are typically included voluntarily and confidentially answered by raters.
Of course, narrative comments can possibly be evaluative, overly critical, or negative, having an adverse impact on acceptability. For example, Smither and Walker (2004) analyzed the impact of upward feedback ratings, as well as narrative comments, over a one-year period for 176 managers. They found that those who received a small number of unfavorable, behaviorally-based comments improved more than other managers, but those who received a large number relative to positive comments significantly declined in performance more than other managers. Like quantitative results, open-ended comments can create strong emotional reactions that can interfere with the acceptance of feedback and lead to diminished engagement and productivity.
In most 360-degree feedback interventions, invited raters are often not given any type of training of guidance to maximize the usefulness of their written comments. Ideally, invited raters should be communicated with early on to describe the purpose of the 360-degree feedback intervention, how their ratings will be used, who will have access to them, and how to make them as behavioral, constructive, and useful for the participant as they can possibly be. This type of communication and training can be helpful to reduce both the leniency and strictness errors and other forms of rating biases.
Coach’s Critique:
One would think that qualitative data would be more beneficial than quantitative data. However, there are pros and significant cons to the outcome of qualitative data. Receiving too much information can have some major pitfalls!
Since questions on the quantitative portion of the 360 are posed in a positive way, participants vulnerability to negative feedback is minimized. As it is, with the quantitative part of the 360, participants often feel overwhelmed by the feedback despite the fact that data is posed in a controlled and somewhat of a positive way (e.g. based on strengths and improvement opportunities). Rather, qualitative data can “open up a can of worms”. Feedback can be negative and harsh. As a result, participant’s esteem to make behavioral changes can actually be shattered!
On the other hand, there may be specific feedback suggestions within the qualitative data that is not covered in the quantitative portion of the 360. Therefore, eliminating qualitative data probably wouldn’t be the best solution. So, if you are a coach or consultant, you play an important role in moderating the effects of the feedback. Here is a suggestion to keep in mind…
You may want to emphasize the themes surrounding the qualitative comments, rather than, focusing on a single comment that might represent a single individual’s experience, perception, or reaction.
What have been your experiences with the effects of qualitative feedback in 360s? How have you balanced between quantitative and qualitative feedback? Any suggestions?
Sandra Mashihi
Dr. Sandra Mashihi is a senior consultant with Envisia Learning, Inc. She has extensive experience in sales training, behavioral assessments and executive coaching. Prior to working at Envisia Learning, Inc., She was an internal Organizational Development Consultant at Marcus & Millichap where she was responsible for initiatives within training & development and recruiting.. Sandra received her Bachelor’s of Science in Psychology from University of California, Los Angeles and received her Master of Science and Doctorate in Organizational Psychology from the California School of Professional Psychology.
We use a 360 degree instrument that is not anonymous as pre-work for our Exercising Influence program. There is an opportunity for respondents to provide comments. Because we have framed it as a way for individuals to provide suggestions as to how the subject could be more influential with them, the comments are generally perceived as valuable insights rather than as critical feedback.