Scorecards & Questions: Best-practice Agency Evaluation Series Part 2

Agency Evaluation Issues

Questionnaires (or scorecards) are one of the most important elements in an agency evaluation program. Designing good questionnaires requires skill and effort but will contribute to a more effective program.

Scorecard at the Heart

The heart of all relation evaluation programs is the questionnaire or scorecard used as the basis for assessment. Problems here include:-

• Uncertainty about the questionnaire or scorecard. Participants do not agree with the evaluation measures or are uncertain about where they came from.
• Too many or too few evaluation criteria. Too many evaluation criteria deters participation. But too few results in an outcome that can be unclear and misunderstood;
• Relevance. Evaluation criteria fail to address the real issues in the relationship, either through wording or prioritization;
• Over-personalized questionnaires or scorecards. So it becomes difficult to compare and benchmark outcomes across different relationships;
• No comments. The program cannot accommodate comments as well as scores. So managers of the program can identify an issue (good or bad) but do not know the reasons for it.

Best Practice Recommendation

Best practice relationship evaluation programs are rooted in questionnaires or scorecards that address real needs. They are developed collaboratively with input from all key stakeholders e.g. Marketing, Advertising, Purchasing and the Agency.

In our experience the best practice questionnaires or scorecards comprise about 15-25 evaluation criteria. This allows an accurate picture of the relationship to emerge without deterring participation.

There should be different questionnaires or scorecards for different agency types – Creative, Media, PR, DM, Design, etc. However, the section headings within the questionnaires or scorecards should be constant across all agency types in order to facilitate cross-comparison or benchmarking of outcomes.

Best practice relationship evaluation programs capture comments. These are especially important if scores are extreme, either very good or very bad. A key goal of a best practice evaluation program is to understand why something is happening. This is helped considerably if assessor comments are gathered and stored.

Finally, the questionnaire or scorecard needs to be prioritized for importance. After all, the final questionnaire or scorecard represents a blueprint of expectations. It is worth remembering the old maxim: “People do not do what’s expected, they do what’s inspected”. Make sure what you inspect in your evaluation program reflects what you want in terms of both content and prioritization.

Source: Decideware Best Practice Agency Relationship Evaluations Whitepaper