“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.”
-Antoine de Saint-Exupery
The usefulness of 360-degree feedback assessments is partly based on the invited rater’s ability to read and understand the questions that are posed. If questions are double barreled (i.e., contain two concepts in one question) it can be confusing to the rater and decrease the accuracy and meaningfulness of the feedback given. Psychometrically sounds 360-degree feedback assessments also measure specific skills or competencies in a way to facilitate behavior change.
The items or questions that compose a skill or competency should be behaviorally oriented and all be related to the concept being measured. If the items or questions “hang together†statistically (i.e., they correlate highly with each other) this is known as internal consistency reliability. Should a single item or question be used to measure a competency (e.g., “The leader listens for understanding†to capture the competency of listening)?
To ensure reliability of a scale, a general rule of thumb is to have a minimum of three behaviors in a 360-degree assessment. In general, the more questions you have for each scale, the more internal consistency and reliability increases. However, when a survey is too long it might decrease the motivation of raters to complete the online questionnaire.
When developing your own customized 360-degree assessment it is important to make sure you have written questions to measure the full scope of the competency you are trying to measure, without having questions that are redundant.
Coach’s Critique:
One of the challenges of writing behavioral items within competency categories is making sure the questions are actionable and truly reflect aspects of the skill area being measured. When writing behavioral competencies, it is sometimes difficult to come up with enough items within a particular competency or having too many. When developing a 360-degree feedback assessment, I found that when I could not come up enough items within a competency, that it generally should not have been categorized as a competency, but rather it could have been rephrased into a behavioral items and placed under another relevant competency. So, in other words, if you can’t come up with enough behavioral items within a competency, then perhaps it might be a good idea to re-evaluate whether the competency should exist as its own category.
Development plans can be either competency or behavioral (item/question) focused so it’s important for coaches to use 360-degree feedback assessments that have clear definitions of the competencies being measured and actionable behaviors for our clients to work on.
As coaches do you:
- Consider the psychometric properties of 360 assessments you purchase from vendors?
- Do you conduct any statistical analyses with 360 assessments you develop for clients?
- How do you share and discuss the competency framework behind a 360 assessment with your client?
- Do you focus on competencies, behaviors or both for developmental planning with your clients?
Sandra Mashihi
Dr. Sandra Mashihi is a senior consultant with Envisia Learning, Inc. She has extensive experience in sales training, behavioral assessments and executive coaching. Prior to working at Envisia Learning, Inc., She was an internal Organizational Development Consultant at Marcus & Millichap where she was responsible for initiatives within training & development and recruiting.. Sandra received her Bachelor’s of Science in Psychology from University of California, Los Angeles and received her Master of Science and Doctorate in Organizational Psychology from the California School of Professional Psychology.
