By Anthony Borrelli
When it comes to performance ratings, the stakes can be high.
Their outcomes can affect promotions, help organizations identify which employees are best suited for training programs, and even gauge a teacher’s effectiveness in the classroom. There’s just one problem: people often rely on generalized impressions or vague memories when completing these questionnaires, which can lead to inaccurate responses. And yet, organizations often rely heavily on this type of feedback.
So, is there a way to make these ratings more reliable? Binghamton University School of Management Associate Professor Tiffany Keller Hansbrough is leading a research team determined to find out, through a series of five studies made possible by a $1 million grant from the U.S. Army Research Institute for the Behavioral and Social Sciences.
Their goal: reduce the gap between what we believe ratings capture and the quality of information they actually provide.
“When the person filling out the ratings doesn’t think the ratings are particularly important, and considering our default way of thinking isn’t always as careful as we’d like it to be, you get a ‘perfect storm’ for inaccurate performance ratings,” Hansbrough said. “Our default way of thinking relies on a gap-filling process, but how do we nudge people into a more careful way of thinking? That’s not something we’re likely to do automatically in these situations, so it’s a tough problem to crack.”
Tiffany Keller Hansbrough, associate professor of leadership and organizational behavior at the Binghamton University School of Management. Image Credit: Jonathan Cohen.
The researchers began by looking at the words used in the evaluation questions and statements, which can sometimes prompt people to endorse behaviors that didn’t actually occur. For example, a question might ask an employee to rate, “My manager wants to know about my career plans.”
The employee might not have an exact memory of the manager sitting down and asking, “What’s your five-year plan?” But because they like their manager, the employee might be inclined to agree with this statement if it seems like something the manager might have done. The opposite can also be true, Hansbrough added, especially if the employee has a negative relationship with that manager.
Questions can also be strategically designed to nudge people to think more carefully about how to respond, Hansbrough said.
Specific questions force the brain to stop and retrieve a particular “video clip” of memory. Instead of relying on a general feeling, raters would access this “episodic” memory, recalling specific instances of behavior.
“Incorporates market trends into project planning” is an example of something more concrete that would move people to episodic memory and reduce gap-filling, Hansbrough said.
Questions like those about specific behaviors also provide leaders with feedback about what they should start doing or stop doing. In contrast, Hansbrough said, vague questions about “leadership presence” offer no clues about how leaders can improve.
When high-performers receive vague feedback about their work, Hansbrough and fellow researchers argue, they become frustrated because they do not see a path to advancement. However, specific details about behaviors provide a clearer path forward.
Hansbrough and her fellow researchers are in the midst of data collection for several studies to test their theory.
“Performance evaluations are super important, and they play out in many different sectors,” Hansbrough said. “Giving people developmental feedback so they can improve is a great idea, but it hinges on the assumption that the data from those performance evaluations is valid and accurate, and this type of feedback hasn’t always lived up to its promise. It’s like a stack of blocks; if it’s not done right, everything collapses.”
