360-degree Agency Evaluation, can you handle the truth?

Using 360-degree feedback as part of your agency evaluation program.

A topic that remains hot in the Agency Relationship Management area is what is commonly called 360-degree Evaluation, which is where an agency provides feedback to their client as part of a formal, structured performance management program.

Introduction

Bill Duggan, Executive Vice President at the Association of National Advertisers (the leading client-side marketing industry body) says that he fields an increasing stream of questions on this complex and emotive topic. The reason, it would appear, is that as agency evaluation & performance management matures as a discipline, clients are seeking to maximize the value of their programs by expanding the focus to include themselves in the equation. This makes perfect sense in the context that both performance and costs can be dramatically influenced by appropriate changes in client processes and behaviors.

However, this is a challenging process to initiate and many organizations, though aware of the benefits, are equally cognizant of the issues that can occur in this highly sensitive area. This paper seeks to explore the entire 360-degree feedback process, looking at the reasons why you should consider implementing such a program, as well as looking at the issues and suggestions on how they can be solved.

Clear and Present Purpose

The most important aspect of a 360-degree Evaluation program is to fully understand why it is being implemented. This is true at all levels of the client organization, and also at the agency. So it is vital that not only the senior executives understand why, but that it is communicated to all participants in the process - this theme of communication and training will be reinforced over and over!

For many organizations the overarching goal is to understand how good a client you are and how you can better “enable” your agencies to operate at their optimum level. Looking at this in more detail there are two key objectives we are looking at.

1.  We need to understand how we can help improve the performance of our agency’s outputs (i.e. better creative, media and other tangible outcomes). This can be summarized as improving the effectiveness of the relationship.

2. Highly topical in the current economic climate is to cut down on costs, waste and friction in the relationship - to better deploy our stretched assets. This can be summarized as improving the efficiency of the relationship.

What is interesting is how many client-side behaviors affect both efficiency and effectiveness, reinforcing what a powerful lever a well managed 360-degree program can become.

The Prerequisite of Trust

Before we look at the mechanics (content, processes and technology) there is a single fundamental building block that must be in place before a 360-degree program should be considered. It can be summed up in one word-TRUST!

In a recent WFA/Decideware survey of agencies 43% of the respondents said the biggest challenge they see with client-agency performance evaluations is providing ‘honest’ feedback on clients during 360 evaluations. 

Without a strong bond of trust and co-operation, both between the agency and client, as well as within the client organization itself, a 360-degree program (no matter how well run) can be a waste of resources and may even end up being counter-productive.

Trust needs to exist at three levels…

•  The agency must trust that the client will use the feedback in a constructive manner. The agency needs to feel that your organization is committed to improving the relationship for the mutual benefit of both parties.

•  The client marketing stakeholders must trust that their agencies are approaching the process with the objective of improving the relationship, not just to release pent-up frustration.

•  The client marketing stakeholders must trust that senior management will use the findings in a positive way, and should thus not view it as a threat.

Communication and training can certainly help build trust prior to a program being implemented, but we would caution for the need to be certain that the client/agency relationship can “sustain” the potential impacts of meaningful feedback.

This may mean that you need to apply the 360 only to selected relationships (if you have a portfolio of agencies). You may want to consider where in the lifecycle the relationship is and be aware that newer relationships may be the most sensitive; therefor the agency is likely to be at its most “diplomatic”. Do be cognizant though that some older relationships may be still sensitive to perceived criticism, as there may never have been an appropriate feedback mechanism in place (“Why are you telling us this now?”). Under no circumstances should a relationship that is “on notice” be included!

Some clients advise that a good idea is to pilot the process with a single strong, healthy relationship. This will allow you to bed down your communication, administrative and analysis processes ahead of roll-out to all agency relationships.

What should we Measure?

Often one of the first questions asked is what should be measured, in terms of client performance - we use the term “performance” loosely and the following section will illustrate why.

The key to the client scorecard is to focus exclusively on enabling processes and behaviors. By this we mean ways in which the client helps the agency perform its designated role.

The criteria being rated should be as clear and focused as possible. Each one should be challenged, asking how a change in this process or behavior affects either efficiency or effectiveness (and hopefully a combination of both).

An aspect we must emphasize is that the agency is not being asked to comment on the client’s marketing/advertising skills. This is a recipe for disaster and is threatening to both the client stakeholders, as well as the agency in having to answer such a loaded question.

A good tip is to have the marketing team help build the criteria on which they are to be evaluated. This has the dual effect of having them perceive the results as more valid, as well as being more committed to act on the feedback. Ask your team to think about this; if they were providing feedback to one of your customers, what types of information they would like and how they would act on it.

One final point is to ensure that the opportunities for improvement are under the span of your control. In other words there is no point in asking a question if your organization either cannot, or is unwilling to make a change in the measured area.

In a recent WFA/Decideware survey, the performance metrics agencies value the most are those that...

1.   Demonstrate the impact on the client’s sales performance

2.   Measure the impact on brand performance

3.   Measure soft metrics like:

•  Collaboration
•  Strategic thinking
•  Client satisfaction
•  Efficiency and effectiveness
•  Compliance

Designing your Process

The favored approach for 360 feedback is to include, for each question, both a rating scale and the ability to collect substantiating comments. Without comments the tool loses much of its ability to generate meaningful development opportunities.

We also advise clients to ask two headline questions (there are many variations on this):

1.  Strengths to Leverage — what is this client team exceptionally good at that aids your performance as an agency

2.  Opportunities for Development — what are the most important areas this client could develop that would better help you do your job as an agency

In terms of the rating process most clients favor a 5 or 9 point rating scale. Some prefer not to use a rating scale at all, simply collect comments, though the general position is that you then lose the ability to pinpoint areas that the agency scores lower or higher.

General advice from the Supplier Performance Management space suggests you should try to get feedback from as many of the agency’s functions as possible. Some examples are...

• Senior management
• Account service team
• Creative
• Financial
• Legal (contractual compliance)

However, in spite of the above point you may well find that your agencies may be reticent to allow individual responses from multiple assessors (and that’s perfectly understandable). What the agency is worried about is that there may be a “loose cannon” that expresses an opinion in an inappropriate way. This could severely damage the relationship — worst case actually losing the business. Until the agency has experienced the process and trusts that the feedback will be used in an appropriate manner, the single feedback approach is common.

One factor to understand is that when assured of confidentiality, agencies will provide much more direct and valuable feedback. There is an interesting technique you may like to consider. Rather than providing the feedback (ratings and comments) directly to your internal marketing stakeholder team, use a 3rd party to review the reports and have that party draw up the suggested action plans.

The 3rd party may disclose some limited aspects of the reports in order to substantiate why the development opportunities were arrived at, but the complete anonymity of the agencies rating and comments are preserved. Some organizations use their internal marketing procurement, operations or agency management functions to act as this 3rd party. Others choose to engage external consultants to act in this “ombudsmen” role.

Understanding the Technology Options

An important consideration is the platform on which to deploy a 360-degree agency evaluation. In the era of email it is highly unusual to hear about paperbased collection so we will concentrate on the three primary modes used…

1.  Many clients use “manual” collection & reporting devices such as Word or Excel.
2.  Some use basic Survey tools to collect feedback (e.g. SurveyMonkey) and then compile reports in Excel or database tools (such as Access).
3.  Others use dedicated agency evaluation / supplier performance management platforms.

The biggest issue for the manual and general survey process is collecting enough valid feedback and compiling scorecard reports in a timely manner, especially as the 360 is done simultaneously with the already complex standard client-on-agency evaluation process. The common frustration is that the administration aspect completely dominates the program, leaving little time and energy for the vital action planning phase.

There are several advantages in deploying a dedicated evaluation platform.

• • Technical administrative horse-power changes the focus from painful collection and data compilation, to meaningful analysis and action planning.
  • Participants are given a level of comfort and have a higher propensity to provide frank input due to the enhanced perception of anonymity and security.
  • They are fully integrated into the existing evaluation process, allowing simple & simultaneous deployment of the 360 along with the existing evaluation program.
  • Advanced reports, both for scores and compiling comments provides meaningful information on which to build action plans, it also allows centralized analysis and provides the ability to look for common patterns and diagnose organizational issues.
  • The technology allows for higher level analytics and year-over-year comparisons. 

Avoiding the Black Hole

One critical and seemingly obviously fact, is that you must be prepared to act on the results.

However, it is interesting that the perception from agencies indicates that occasionally the feedback they provide seems to disappear into a black hole. To this end there are four key communication elements to address...

1. Engage senior leadership from both the agency and client side.
2. Always respond to the agency to let them know you are reviewing their feedback.
3. Conduct debrief meetings where results are shared and actions plans are set in motion.
4. If you are not going to take action on an element of the feedback (for whatever reason), ensure that you communicate this to agency so they understand that they their voice has been heard.

There are several aspects that need to be understood in the action planning phase and it is suggested that a formal training / communication strategy is included in your program. This should include…

• • How to identify priority processes & behaviors for improvement
  • How to create an action plan
  • How to identify and access development resources

In many cases the action is a subtle tweak, rather than completely re-engineering a client/agency process from scratch. This requires a diagnostic approach to review carefully the nuances of the feedback you are reviewing. It is also useful to read the 360-degree Agency-on-Client reports side by side with the traditional Client-on-Agency reports. Often you will find rating and comments tell different sides of the same story.

When you are reviewing the feedback provided you should be cognizant that some agencies may not be as frank as others. To adjust for this, when looking at the ratings, don’t worry about absolute scores, just look where the dips are. So as an example (on a scale of 1-5) if one agency rates your team as 5,5,4,5 on a set of attributes, this may mean exactly the same as 3,3,2,3 to another.

Also expect “sugar-coated” comments! You may need to read between the lines when interpreting the verbatim to discover the true intent. One amusing characteristic you may find is the classic two-part comment, for example “You provide excellent feedback, when we are able to get hold of your senior management”. You will probably find that the comments will become more direct after the first time around as agencies get more comfortable with the process.

Given the different levels of agency propensity to be forthright, it is also vital that you DO NOT COMPARE overall feedback scores across your teams and use those scores as a basis for internal ranking or performance measurement. This will break one of the fundamental pillars of trust and can be extremely harmful for your 360 process.

Finally it is essential that both the stakeholder and agency take ownership of the action plan.
An action plan is only as useful as its execution!

Conclusion

In this paper we have explored all aspects of the 360-degree feedback process. We put forward the case that if incorporated into a robust evaluation program, the 360 can be one of the most powerful tools we have available to improve client/agency relationship performance, minimize inefficiencies and reduce costs.

Certainly it is a sensitive area and if handled incorrectly can cause more harm than good. Without a collaborative and positive approach it can degenerate into finger-pointing and ultimately lose any value it could generate.

However, with a foundation of trust, a healthy dose of training & communication and supported by appropriate processes & technology, this is an exciting and innovative way to dramatically affect the power of your marketing effectiveness.

Reference Sources

• It’s a Two-Way Street – Agency experts give their perspective on the client-agency relationship evaluation – ANA 2019
• Client-agency performance evaluations: the agency view – WFA 2020
• Agency Evaluations: What’s Good, New, Next - Joanne Davis Consulting Inc. — ANA 2007 ANA Agency/Client Forum
• Hewlett-Packard gets the most of its supplier feedback:
  By William Atkinson — Purchasing, 4/10/2008
• Strategic Supply Management: Robert J Trent: 2007
• Guidelines for MultiSource Feedback When Used for Decision Making : David W. Bracken - dwb assessments, inc. Carol W. Timmreck - Shell Oil Company: 1999
• Are you really a customer of choice? by Rob Handfield and J Lehr McKenzie (CPO Agenda: Winter 2008-09)
• Managing Performance to Maximize Results: Harvard Business School Press (Should you use 360 feedback for performance reviews? - Edward Prewitt + The ratings game - Lauren Keller Johnson)