Agency evaluation programs are important but they do have challenges. One of the most important is...
There are key elements that advertisers should consider when developing agency evaluation programs. Frequency, assessment models, and rating scales should all be carefully considered and developed to ensure the success of the program.
Comments should stathe the Issue, Desired State and Action.
How Often?
Advertisers have several options for how often they run evaluations throughout the year. Larger advertisers with several agency relationships typically run one annual evaluation. For advertisers at this large scale (often global, multi-brand companies), it is a pragmatic approach because by the time they have set up the evaluation, run it, performed all the analytics and delivered against the action plans, a year has passed, and it is then time for the next evaluation. However, many advertisers feel that one annual evaluation is not enough and issues that come up during the year need to be uncovered and addressed. Therefore, they run two evaluations per year. Other advertisers run quarterly evaluations, but these tend to focus on more tactical vendor relationships, where KPIs and SLAs are often part of the contractual agreement.
A relatively new trend seeks to balance the need to lighten the administrative and survey load (using an annual evaluation), and at the same time provide the benefits of the twice-yearly evaluation. This is achieved by moving to a full scale annual evaluation on all critical agency relationships, followed by a mid-year “pulse check” evaluation. This is particularly true for larger advertisers where it is difficult to run multiple evaluations in one year. Whereas the annual evaluation centers on the relationship and the work, the mid-year pulse check contains only a few questions and typically focuses on the action plan that was put in place during the annual evaluation. This pulse check allows you to ask top line questions to ensure that the action plan is on track, and if not, the ability to adjust things to ensure that the goals are met.
Assessment Models
Advertisers have various choices for the type of assessments they want to run involving both the client and agency. The most common are:
- a one-way evaluation (where the client evaluates the agency’s performance)
- an agency self-assessment
- an agency-on-client “360”, where an agency provides feedback on their client
The evaluation including an agency-on-client “360” has typically been described as the gold standard; however, there can be some issues with this model. Sometimes agencies find it difficult to comment directly and candidly on the marketing team and as a result the feedback is candy-coated and difficult to interpret. The agency must trust that the client will use the feedback in a constructive manner, and feel that the organization is committed to improving the relationship for the mutual benefit of both parties.
Due to these challenges, some advertisers are trending towards implementing a “Reflective Self-Assessment” evaluation. In this type of evaluation, the advertiser evaluates the agency and then the agency provides a self-assessment. Key to this model is the self-assessment where the agency is asked to take into consideration how the client has impacted their ability to execute, and what the client could to assist them improve. This less direct approach allows the agency to provide feedback in a more passive, less threatening manner, thus allowing them to provide more insightful and actionable comments. For example, if the agency believes that they were not able to respond to the brief properly because it was lacking some important information, they can provide a lower rating in the self-assessment on the “Creative is on Brief” question, and then comment on how the advertiser could help by improving the quality of the brief in order for the agency to provide quality outputs.
Rating Scales - Rise of the 5-star scale
Every day, consumers are bombarded with rating scales. When we make a purchase on Amazon, research restaurant reviews, or choose a hotel, a “5-Star” scale is most often used as part of this research.
A question that is increasingly asked is whether agency evaluations will adopt this idea, using a rating scale that people now naturally understand and use in the consumer world. Most people implicitly understand what a 3, 4 and 5 star rating mean when booking a hotel. Furthermore, if there is a dispersed geography, this 5-star scale is consistently understood in all regions and countries.
Currently many advertisers utilize a research based standard 5-point scale (using descriptors like “Meets Expectations”, “Exceeds Expectations”, “Does not meet Expectations”), or scale that is in line with their HR efforts. In this case, it is important that the rating scale is balanced and does not lead to “bunching”. An example of bunching would be where a 10-point scale is used, and most of the participant survey ratings are 9 or 10, limiting the ability to differentiate between high and low performing agencies. Guidelines and definitions are also used to ensure that everyone understands the definition of each rating.
For additional information regarding Agency Evaluation, please visit www.decideware.com.
