 “We now have what I like to call the buffet table of choices; we are customizing to our specific needs.” -Pamela Peeke
When looking for the appropriate 360-degree feedback tool, many people are often uncertain about whether they should use a customized 360-degree feedback tool or just buy a vendor’s off-the-shelf tool. Most people think that if the 360 is customized, then it must be more relevant and valid.
A well-designed psychometrically-sound off-the-shelf 360 degree feedback questionnaire can be used if the competencies being measured and the behaviors are relevant. For example, if an organization is seeking to assess executive leadership behavior, a valid and reliable off-the-shelf tool would be sufficient and more feasible. On the other hand, if behaviors being measured are not relevant to the organizational culture, then completing the 360-degree feedback questionnaire could be difficult and will not give you the results you are looking for. A customized 360-degree feedback assessment can be useful if designed to fit your exact needs or to support unique organizational competency frameworks or strategic development objectives and initiatives.
Coach’s Critique:
As usual, an individual called in today and asked about our 360 degree assessment and wanted to know if it’s better for her to move forward with a customized tool or one of our off-the-shelf tools. We told her that in order for her to make the right decision about this, we would need her to answer the following questions:
1. Do you already have an established and unique competency model for the target audience you want to assess in your organization?
2. Are you using a 360-degree feedback process for a few people or the entire organization?
3. Is your competency model ready and established enough to be placed into a customized engine, or do you have to improve and fine tune your current competency model?
4. Is the cost of creating a customized tool, which may include internal or external consultants helping to create a robust competency model, worth it if the competencies and behaviors you are trying to assess are close enough to be revealed through an off-the-shelf tool?
5. What specific behaviors are you looking to assess (i.e. what do you want talent to be doing more, less, or differently)?
6. If you create you own competencies and develop a 360-degree assessment around it, what plans do you have in place to ensure it has face validity and reliability?
Upon asking these questions to help this individual make the right decision, we advised her that if she finds a tool that closely assesses the competencies and behaviors she is looking to assess, and if the tool has some level of sound psychometric properties (most vendor tools should have some level of reliability, although a tool that has both reliability and validity is preferable), then it might be a better decision to purchase one. However, if there is a set of behaviors that is unique to the organization that she is trying to assess, then it might be a good idea to create a customized tool and design a unique competency model for the organization.
What have you your experiences been with using off-the-shelf vendor tools vs. customized tools?
Sandra Mashihi
Dr. Sandra Mashihi is a senior consultant with Envisia Learning, Inc. She has extensive experience in sales training, behavioral assessments and executive coaching. Prior to working at Envisia Learning, Inc., She was an internal Organizational Development Consultant at Marcus & Millichap where she was responsible for initiatives within training & development and recruiting.. Sandra received her Bachelor’s of Science in Psychology from University of California, Los Angeles and received her Master of Science and Doctorate in Organizational Psychology from the California School of Professional Psychology.
