“Some people see things that are and ask, Why? Some people dream of things that never were and ask, Why not? Some people have to go to work and don’t have time for all that.”
George Carlin
To stimulate research on the topic of poor leadership, Robert Hogan in 1990 suggested that the base rate of leadership incompetence was between 60% and 75%. Other research has confirmed that approximately one out of two executive leaders fail in corporate America. These leaders seem to consistently lack emotional intelligence, be overly controlling, poor at delegation and problem solving and perceived to be untrustworthy. Leaders may be made but clearly finding the ones that are born with “leadership set points†would appear to maximize the success of organizations.
Based on several recent surveys (e.g., Abderdeen Group and Rocket-Hire), the utilization of assessment tools for pre-employment selection and promotion is approximately 60% to 70% across all industries with some projections of increased use in the next 12 months of about 14%.
Of those using pre-employment assessments across job levels, the most popular approaches continue to be evaluation of work history, candidate interviews, skill and aptitude tests, and personality inventories. Which approach to measuring key aspects of potential talent’s knowledge, experience and competence actually do a good job of predicting future success and performance? The table below summarizes a large number of recent meta-analytic research studies in the industrial/organizational psychology literature across diverse industries, job levels and different measures of job performance and success.
The numbers in the column are called validity coefficients and they can range from 0 (no association with performance outcomes) to 1.0 (perfect association). This table is based on very diverse industries for leadership positions across all levels using a wide variety of specific measures of success, salary, promotion, and performance in mind. In the industrial/organizational literature, it is not unusual to find that most assessments today are only modest at actually predicting future success of leaders in any industry—regardless of how it is defined and measured. The use of two or more of these pre-employment and selection methods doesn’t dramatically increase predictive validity but relying on only one approach may lead to erroneous hires and selection decisions (e.g., relying only on personality inventory results or interviews).
Any of these methods can be used for selecting talent as long as they are based on a systematic job analysis summarizing relevant knowledge, skills and abilities required for successful performance. There are many ways to establish validation of a pre-employment assessment with the most common methods used including content validation (showing a link between the job requirements and the content of an assessment) and criterion related validation (showing a statistical relationship between the assessment and some measure of performance) methods.
What We Know About Selecting Successful Leaders in the Future
A number of practical and important observations can be made by looking at the relative average predictive validities (correlation coefficients) ranging from the highest (.54 for work samples) to the lowest (.01 for age).
- No specific assessment approach is statistically very strong in predicting success or performance although work sample simulations, cognitive ability tests for entry positions and more structured interviews are the strongest.
- The standard “mutual seduction†interview is probably only modest at predicting future performance. Predictive validity increases as interviews become more structured, based on a comprehensive review of the position requirements and based either on specific situations that can be reliably evaluated or behavioral samples from the past.
- Interests are relatively weak at telling us much about future performance or competence (just watch the television show American Idol and you can see this in action). However, interests are strong predictors of job satisfaction and turnover so it is important to get a sense of what will maximally engage talent. New research suggests that job specific interests might be stronger in predicting performance and turnover.
- Although reference checks have legal restrictions that minimize their usefulness, in concept they should be pretty revealing if you can get information to be shared by previous colleagues, peers and employers given the predictive power of peer and supervisory ratings.
- Minimize stereotypes about the value of talent with particular educational backgrounds and age—both are virtually useless for making predictions about future leadership success.
- Those who demonstrate cognitive ability seem to learn more rapidly and assimilate information. Cognitive ability tests are known for potential adverse impact and are often viewed more negatively by prospective candidates. At higher levels there is a compression of mathematical-logical intelligence making these type of assessments less useful (predictive) but still the more of it you have the greater the job performance.
- It’s not how smart you are but how you are smart. Interpersonal competence, self-awareness and social awareness (ingredients of emotional intelligence) are probably better predictors of who won’t succeed than who will. Be careful about overstated claims about the predictive power of emotional intelligence on job performance.
- “Show me†assessments or simulations appear to be universally strong predictors of leadership success. These types of measures have been incorporated into assessment centers with strong predictive validity and little or no adverse impact. Because they are designed around the job in question, candidates also respond much better than to pre-employment approaches that don’t appear to be immediately relevant to the position (e.g., intelligence tests, personality inventories).
- Personality measures are modest predictors of job success with two “universal†or generalizable factors typically found to have the strongest association with job performance across diverse settings: conscientiousness (driven, dependable, organized, achievement oriented, responsible) and emotional stability (self-confident, even tempered, adaptable, resilient, emotionally well adjusted).
- When job performance depends on leading and influencing (e.g., sales and managerial positions) extroversion is a significant personality predictor and for customer service oriented positions interpersonal factors (agreeableness) appears most strongly associated with performance and success.
- In positions requiring creativity and innovation, a personality factor often referred to as “openness to experience†is quite predictive.
- There isn’t much argument that selecting and promoting the best leadership talent is a strategic competitive advantage. However, what approach to use for “human handicapping†is an important decision when companies, large and small, begin to introduce specific assessment methods for pre-employment hiring and promotional decision making. Most are significantly better than chance and some are certainly better at increasing the odds of predicting high performers.
If only it was as easy as the final scene in the baseball movie called The Natural where Roy Hobbs breaks his precious bat and tells the bat boy “Pick me out a winner Bobby” with pretty dramatic results (and even better score by Randy Newman)….Be well….
Ken Nowack
Kenneth Nowack, Ph.D. is a licensed psychologist (PSY13758) and President & Chief Research Officer/Co-Founder of Envisia Learning, is a member of the Consortium for Research on Emotional Intelligence in Organizations. Ken also serves as the Associate Editor of Consulting Psychology Journal: Practice and Research. His recent book Clueless: Coaching People Who Just Don’t Get It is available for free for a limited time by signing up for free blog updates (Learn more at our website)
