Decideware

Understanding and interpreting extremes in scores, often by only one assessor

In analysing performance reviews perhaps one of the greatest conundrums we are faced with is how to interpret results where a problem is confined to one or two people, rather than being systemic and question related.

How should we interpret an assessment that is wildly different from the norm of all other assessors - whether high or low – and what should we do with this “Rogue” assessor, when it’s very different from the rest?

In addressing the title of this article, the 1960’s revival of an old sea shanty “What shall we do with the drunken sailor……early in the morning?” came to mind. Some of the solutions from the song are definitely inappropriate. For instance, we should not:

• “……Sling him in the longboat ‘til he’s sober” (Cast him adrift, isolate him);
• “……Hit him on the head with a broken hammer” (Punish him);
• “……Send him down to Davy Jones Locker” (Shoot the messenger);

This is not market research!!!

So how should we treat these “Rogue” assessors? Market researchers and statisticians, when dealing with such anomalies in a large-scale survey would regard them as normal sample error and either discard them, or ignore them – both high and low.

However, in an Agency or Supplier Evaluation we mostly work with much smaller numbers and the qualitative information gathered is as important as the ratings results in helping us to determine action plans to improve performance, so discarding or ignoring an assessment without understanding why should not be an option to consider.

1. Identify

The first step that we, and our relationship consulting partner Navigare, would take is to identify the “Rogue” through the Decideware’s Relationship Optimizer “Cross-tab” report (only available on the client side to clients and for internal use only). Our search then is all about looking for signs of three things: Malice, mischief or mismanagement.

2. Confirm, explore and understand

Then we would suggest that a neutral party talk with the person, to confirm their views, better understand their situation and perspective on the score and to gain additional insight into the assessment and seek a further explanation and reasons for their rating.

It may also be necessary to talk to their manager and the supplier manager, to understand their perspective on both the person and their assessment and also their relationship with the business partner.
If you decide to do this, then the individual’s permission should be sought to talk to their manager and disclose their identity and views.

We would advise that the “Rogue” assessor’s actual ratings should not be disclosed, merely the fact that they differed significantly from the norm (this is to conform with Privacy principles and, in our case, comply with the ethics of Market and Social Research, that prohibits disclosure of an individual’s response without their express permission).

3. Involve and engage

Finally, in a step which is sometimes missed when action plan improvements are considered, involve the whole team in reviewing and interpreting the survey results. It is this opportunity for engagement that will allow for extremes in views to be discussed, considered and resolved.

Some motivations to look for in “Rogue” markers

From our experience with Human Resources performance reviews and 360º feedback surveys, we know there are a number of lessons that can help us to understand people and corporate dynamics, and sometimes perhaps explain the reasons for higher, or lower, scores in any assessment process.

Sometimes it is not what people are saying, but who they are and why they are saying it, that helps us to interpret results for our clients. For example, in Human Resources, it is often said there are three types of assessors;

• Those who want to be famous and to be seen by their managers as being “tough markers” and looking after the company’s interests by enforcing their sometimes uncompromising or unreasonable expectations. Such people tend not to worry about how they are viewed by their peers, associates, suppliers or other stakeholders as they often have higher ambitions;
  
• At the other extreme there are those “soft markers” who want to be loved and go out of their way to avoid confrontation of any type;
  
• Thankfully, there are the great majority who are fair, open and honest in their assessment and who only seek to improve relationships and performance for the common good. They provide candour and transparency and expect it in return.

Added to that are two other types that may not apply to assessing business partner relationships, but whose assessment style may sometimes be motivated by other intrinsic reasons: Some are often hard markers to their peers, because they compete with them in their careers, whilst some direct reports can often be vindictive against a manager and wish to “get even”.

In assessing business partner relationships, we find that the “tough markers” are more typically found on the client side and suggest that some motivations are:

• Arrogance: The client/supplier relationship can often be seen by some as a master/servant one, rather than a strategic partner  - particularly where a client is a blue chip company or brand. This can be a major problem when tolerated by the culture of a Company and coupled with a “tough marker” wanting to be famous and fast-track a career.
• My Way: Non-team players, or those with a grudge/chip on one or both shoulders, can sometimes have very strong views on the way they think things ought to be done and reflect this in their business relationships and assessments;
• Cry for help: The opposite of “My Way” is the person who genuinely cares but is frustrated by their manager when improvements/ideas are raised, and sees the anonymity of the survey as a way of raising their argument to a higher level;
• New Broom: New appointees often come to a role/organisation with a different set of expectations and standards. These may be refreshing and reflect reality, or on the other side of the ledger, reflect a poor understanding of their new company/role or may just be a way of quickly making their mark.
• Honest: Let’s not forget that the tough marker may also be the only one that has the intestinal fortitude to call a spade a spade. They may be right.

On the other hand, soft markers can in fact be counter-productive to a relationship, through their lack of honesty and objectivity. Their motivations are more often about wanting an easy life and/or needing to be loved or recognised and that their contribution or role is important. This type of assessor is probably well settled in the role, keeps their head down, dislikes confrontation – but loves the attention (and lunches, entertainment, sports tickets etc) and won’t make waves.

So in summary...

To end the nautical theme, the last thing that any business relationship needs is a loose cannon or a drunken sailor adrift in their midst. So we suggest it is vital that three steps be taken with a “Rogue” marker:

• Identify the individual;
• Confirm, explore and understand their point of view;
• Avoid a “top-down” approach to action plans. Involve and engage, perhaps workshop with a facilitator, the teams to get their buy-in on solutions and follow up actions.

Author: Ron Latham (Latham Consulting)