“In his later years Pablo Picasso was not allowed to roam an art gallery unattended, for he had previously been discovered in the act of trying to improve on one of his old masterpieces.â€
Unknown
Getting people and leaders to change is pretty important—particularly enhancing behaviors associated with increased effectiveness.
The “magnitude†of change or “degree†of change seems pretty important for coaching and talent development. In science, we measure this amount of change statistically with something called effect sizes. Approximately half of all published significant effect sizes in psychology are equal to or greater than .24 ((Hemphill, J. (2003). Interpreting the magnitudes of correlation coefficients. American Psychologist, 58, 78-80)). In a practical sense, these sizes are relatively small.
So, just how “weak†are 360 feedback and training interventions for impacting behavior change?
WEAK FORCE #1: 360 FEEDBACK
Research by Smither (2005) on 360 feedback suggests that although feedback does result in significant performance improvement in a review of 24 longitudinal 360 studies, effect sizes are relatively small–.05 for peers and .15 for supervisors and direct reports ((Smither, J., London, M. & Reilly, R. (2005). Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings. Personnel Psychology, 58, 33-66)). These findings suggest that “zebras don’t easily lose their stripes.â€
It would appear that we must accept that all of us have some skill and ability “set points†that may provide an upward ceiling to the growth and development of most individuals. One could also use these findings on the contributions of genetics to personality and leadership emergence to make a sound argument that “leaders are made†and not born but it would appear that both genetic predisposition and environment interact together to shape the overall development of key skills and abilities that impact our client’s professional and personal successes and failures in life.
People do change following feedback but the magnitude of change might be quite small and practically insignificant. At best, 360 feedback might be categorized as a weak force intervention.
WEAK FORCE #2 MANAGEMENT TRAINING
The 2014Â State of the Industry Report by the Association for Training & Development (ATD) estimates that companies in the US spent over $164 billion on employee learning and development in 2013. In 2013, organizations on average spent $1,208 per employee on training and development. This is a small 1 percent increase over last year (an additional $13 more per employee). The number of learning hours used per employee also slightly increased to 31.5 hours from 30.3 hours.
Marshall Goldsmith reviewed how well 86,000 leadership training participants actually learned from the experience. He found that the people who went home, talked about the learning and worked deliberately to implement new behaviors learned best. But those who just went back home and did no follow-up showed no improvement at all.
A recent illuminating study by Taylor and colleagues who conducted a new meta-analysis of 107 studies on the effects of management training on training transfer–that is ratings of leaders’ on the job behavior viewed by their bosses, direct reports and colleagues ((Taylor, P., Russ-Eft, D. & Taylor, H. (2009). Transfer of management training from alternative perspectives. Journal of Applied Psychology, 94, 104-121)).
In all analyses they conducted, self-ratings were terribly inflated relative to others who provided ratings. When training was focused on general management skills, the effect size for each of the rater groups was: Self (.72), Boss (.53), Peers (.33) and direct reports (.11). For training focused on just goal setting and performance appraisal skills: Self (1.55), Boss (no studies), Peers (no studies) and direct reports (.33).
In a special analysis of 14 studies that had ratings from the participant, his/her manager, direct reports and peers found the following effect sizes (small effect size is about .2, moderate about .5 and large about .8 or higher) for training focusing on enhancing interpersonal skills:
Self Ratings .50
Boss Ratings .33
Peer Ratings .34
Direct Report Ratings .04
As they authors point out, “Our most surprising–and disconcerting– finding was the relatively small average effect sizes for the transfer of interpersonal skills training, the predominant management training topic, derived from subordinates’ ratings.”
They were being liberal to suggest that the effect size they found (.04) could even be considered “relatively small.” What this finding really suggests is that direct reports really didn’t see any meaningful changes in leadership behavior in one of the most common reasons for providing leadership training. Direct reports seemed to observe some changes when the focus of training was based on goal setting and performance appraisal but  hardly any when it was focused on “general management.”
Ok, at least we can say that the use of self-ratings in evaluation of leadership training transfer may be a bit self-inflated. If you want to argue that training is actually resulting in new behaviors back on the job you would have to really convince me that direct reports don’t really have a great perspective about leadership practices of their bosses. As they say, “people today don’t leave organizations they leave bad bosses.”
Are there ways of increasing the potency of training?
Practice Under Pressure Makes Perfect
Essentially, training is intended to help people develop new habits and enhance effectiveness in specific skills. In order to do so, repetition is important. Also, it is important to allow time to develop and integrate the new habit in one’s daily routine. A week long leadership program is unlikely to lead to the formation of new habits. Initiating behavior change is hard and sustaining it over time is even more challenging. Encourage leaders to practice new behaviors back on the job–under the pressure situations they face every day.
Consider Different Learning Styles
In my research I have seen hundreds of people read books and learn nothing. Not everyone learns the same way. Consider blended learning approaches to ensure that everyone has an equal opportunity reflect, learn and apply information and skills.
Avoid Case Study Overload
Leadership development that is predominantly using a case study approach may stimulate problem solving and analysis but it certainly won’t teach leadership skills. Leadership development is about enhancing specific skills and behaviors—you can do case studies all day and not be more competent in what leaders actually are required to do all day.
Evaluate Your Program
It’s great that the leadership participants liked the facilitator and material. More important is whether anyone notices actual behavior change after the leader leaves the training. If you have to use “happy face†evaluations, at least use a “post-then†approach to enhance the validity of your subjective evaluations. Never heard of “post-then†evaluations? That’s one of the reasons evaluations of leadership development programs are weak or never go beyond “level 1†approaches.
Hold Managers Accountable to be a Coach
If the participant’s manager isn’t involved in the leadership initiative then you have a weak program. Managers of program participants minimally need to share the purpose and goals of the program, clarify expectations and hold the participant accountable to put to together a learning development plan to apply and practice one or more skills taught in the program.
Seek Mentoring and Coaching for Program Participants
Peer coaching and/or mentoring can be incredibly valuable to amplify and accelerate learning from leadership development efforts. Assigning a peer coach from the program or organizational mentor for each participant can be useful to continue skill practice and discussion outside of the leadership program.
Provide Actual Organizational Problems as Projects
Experience is the best teacher. Provide actual organizational problems for leaders to solve in small or large groups as part of your leadership development effort. The transfer of learning is stronger than abstract concepts or case studies so commonly used in most leadership training programs.
Help Executives See Themselves Accurately
We have published research supporting the concept that most leaders have inflated views of their strengths. Incorporate multi-rater or 360 degree feedback assessments in your leadership development efforts to help leaders compare self-perceptions to those of other key internal and external stakeholders. Emphasize the strengths of leaders and encourage behavioral action plans following feedback. Feedback as we have seen is a necessary, but not sufficient, condition for behavior change. Hopefully, 360 feedback will at least lead to greater self-insight and awareness and when things go ideally we can expect commitment to deliberate practice resulting in increased effectiveness and performance.
Focus on Leader Behavior and Not Leadership Per Se
Daniel Goleman suggests that 50% to 70% of the culture of a team or organization is directly attributed to the leader’s behavior. Our own research suggests that leaders play the strongest role in creating a psychologically healthy climate ((Nowack, K. (2006). Emotional intelligence: Leaders Make a Difference. HR Trends, 17, 40-42)). Focus on behaviors that can be deliberately practiced to enhance effectiveness.
Keep the relative impact of these two relatively weak force interventions of 360 degree feedback and training interventions in mind the next time vendors try to sell you solutions for the wrong problem or suggest they can convert competent jerks into lovable stars.
As Robert Gallagher observed, “Change is inevitable – except from a vending machineâ€â€¦.Be well…
Ken Nowack
Kenneth Nowack, Ph.D. is a licensed psychologist (PSY13758) and President & Chief Research Officer/Co-Founder of Envisia Learning, is a member of the Consortium for Research on Emotional Intelligence in Organizations. Ken also serves as the Associate Editor of Consulting Psychology Journal: Practice and Research. His recent book Clueless: Coaching People Who Just Don’t Get It is available for free for a limited time by signing up for free blog updates (Learn more at our website)
The discussion omitted consideration of two variables: the duration of time between the intervention and the post-assessment, and the sensitivity of the observers engaged in reassessment. It is now generally acknowledged that small, incremental changes have the greatest probability of resulting in meaningful, sustainable change over time. Such small course corrections, measured too early however, may not impress observers as substantial. Additionally, the vested interests of a given observer group (e.g.,direct reports) may lead to an under-reporting of small but meaningful changes that do not conform to that groups expectations of desirable change.
Both these issues speak to the value of making explicitly public the changes one is targeting. Otherwise, the “radar” of the observers may not be properly calibrated to assess the actual effect (remember the frog in the saucepan).
The value of follow-on coaching, mentioned in the article, should be emphasized. The time-honored consultant failure of providing an inspiring dog-and-pony show without follow-up to maintain the change trajectory may finally (and thankfully) be facing its demise.