“I was afraid of the internet… because I couldn’t type.” -Jack Welch
The two most popular approaches for data collection are online or paper-and-pencil surveys. While, the majority of vendors and companies today utilize Internet-based administration systems, several studies have verified that there is equivalence in using online methods and handwritten tools. For example, Penny (2003) ((Penny, J. (2003) Exploring differential item functioning in a 360-degree assessment: Rater source and method of delivery. Organizational Research Methods, 6, 61-79.))Â and Smither, Walker & Yap (2004) ((Smither, J.W., Walker, A.G., & Yap, M.K.T. (2004) An examination of the equivalence of web-based versus paper-and-pencil upward feedback ratings: rater -and-ratee-level analyses. Educational and Psychological Measurement, 64, 40-61.)) found no significant differences between the two formats. These studies were based on a sample of 5,257 employees, and 374 groups of 360 participants.
Coach’s Critique:
Although some researchers have found that on-line tools and paper and pencil tools have no significant difference, I tend to believe that the fact that scoring can be more easily automated is what makes up for the difference between the two tools. Imagine implementing a 360-degree feedback process on an entire organization…scoring is obviously challenging, and as a result this can lead to more errors. So, wether or not there is a difference between the two tools in terms of how they affected feedback, it seems that any tools that is prone to errors can diminish the effectiveness of the tool. While many people may still prefer to use handwritten tools, it seems important to modernize the approach, and utilize an online tools which can track, store, and protect information in a more reliable way.
What are some of your thought about on-line vs. handwritten 360-degree assessments? Do you believe there are advantages to using handwritten assessments? If so, what are they?
Sandra Mashihi
Dr. Sandra Mashihi is a senior consultant with Envisia Learning, Inc. She has extensive experience in sales training, behavioral assessments and executive coaching. Prior to working at Envisia Learning, Inc., She was an internal Organizational Development Consultant at Marcus & Millichap where she was responsible for initiatives within training & development and recruiting.. Sandra received her Bachelor’s of Science in Psychology from University of California, Los Angeles and received her Master of Science and Doctorate in Organizational Psychology from the California School of Professional Psychology.
