“A child of five would understand this. Send someone to fetch a child of five.”
Groucho Marx
There are six important factors that most vendors won’t tell you about that directly impact the effectiveness and success of multi-rater feedback (see my last Blog for the first three). In Part II of this Blog I will continue to explore the last three in more detail.
1. Perceptual distortions by participants and raters make interpretation of 360-feedback results challenging.
In our own coaching practice, using diverse multi-rater assessments measuring different competency models, we have repeatedly observed that under-estimators (those whose self-ratings are meaningfully lower than others) tend to be highly perfectionist, self-critical, overly achievement striving and likely to focus on their perceived weaknesses rather than leveraging their “signature†strengths in developmental planning discussions.
Despite, trying to help our coachees interpret the feedback findings in a “balanced†manner, these over-estimators appear to be hypervigilant to the perceived “negative†information contained in their report and often “fixate†on the lowest average scores on ratings scales and the open-ended comments that appear to be “neutral or critical†in tone relative to other more positive comments collected within rater groups.
Our observations support prior research showing that individuals with negative self-views (e.g., those who are depressed) tend to tune into feedback that portrays them critically as opposed to positively (Giesler, Josephs & Swann, 1996).
2. There Might be Limits to the Magnitude we can Expect Leaders to Actually Change Behavior Following Feedback.
We have already noted that Smither et al. (2005) found that although feedback does result in significant performance improvement, effect sizes are relatively small suggesting that “zebras don’t easily lose their stripes.â€
Arvey and colleagues, based on twin studies, estimate that about 33% of the variance in holding leadership roles across diverse organizations can be attributed to genetic factors (Arvey, Zhang, Avolio & Kreueger, 2007) and findings from numerous studies of personality show that genetic effects account for approximately 50% of the variance in five factor model domains (Bouchard & Loehlin, 2001).
It would appear that we must accept that all of us have some skill and ability “set points†that may provide an upward ceiling to the growth and development of many coachees.
3. Feedback Combined with Structured Follow Up and Coaching Leads to Better Performance Outcomes.
All too often, vendors and some practitioners espouse the “diagnose and adios†approach to multi-rater feedback hoping that self-directed insight alone will result in motivated behavioral change efforts.
In one of the few empirical studies recently conducted on the impact of executive coaching, Smither et al. (2003) reported that after receiving 360-degree feedback, managers who worked with a coach were significantly more likely to set measureable and specific goals, solicit ideas for improvement and subsequently received improved performance ratings. Thatch (2002) found that in six weeks of executive coaching following multi-rater feedback, performance increased by 60% and in a much cited study in the public sector, Olivero, Bane & Kopelman (1997) found that employee feedback and coaching for two-months increased productivity above the effects of a managerial training program (22.4% versus 80.0%) for 31 participants.
These coaching studies all support the importance of supportive follow-up after feedback is received to help facilitate developmental action planning and practice of targeted behaviors.
Well, there you have it. Secrets from a vendor about other vendors and what none of us will tell you about 360 feedback tools! However, now that you know about these five factors, I’d like to share one last secret–What matters most about muti-rater feedback is the process and not the tool.
Do it right or don’t do it at all…..Be well…..
Ken Nowack
Kenneth Nowack, Ph.D. is a licensed psychologist (PSY13758) and President & Chief Research Officer/Co-Founder of Envisia Learning, is a member of the Consortium for Research on Emotional Intelligence in Organizations. Ken also serves as the Associate Editor of Consulting Psychology Journal: Practice and Research. His recent book Clueless: Coaching People Who Just Don’t Get It is available for free for a limited time by signing up for free blog updates (Learn more at our website)
