Logical Benchmarking

“The key to good decision making is not knowledge. It is understanding. We are swimming in the former. We are desperately lacking in the latter.”

— Malcolm Gladwell

So you’re eagerly anticipating receiving scores from your latest marketing research study which you’ll use to guide a series of next step decisions to get your product to market.  But, have you thought about what benchmarks you’ll consider to help you UNDERSTAND whether you should “go” and move forward, or “no-go” and head back to the drawing board?

I’m often asked by my clients, “What is a good score on this measure?” or “What score do we need to feel confident that we can move forward?”

Well, most marketing researchers will answer “It depends,” which while technically correct, provides zero value to the marketer or product developer.

So, at the risk of stirring up a serious geek-debate among my fellow researchers out there who would love to take this from the practical to the academic, let me provide some generalized benchmarks you can use to understand if you are being shown the green light to move forward or a red flag to stop and rework the next time you get back concept or product testing scores. 

Please note, earlier I said, “It depends” is technically the correct answer for the question of benchmarks because every case is different.  But, the following benchmarks are based on standard CPG marketing research scales and a certain amount of conventional logic; they are not meant to be written in stone, and should only serve to provide a context for guidance and decision-support.

Common Logic-Based Success Benchmarks & Red Flags

• Likeability (5-point scale)

  • Benchmark: 70% Top-2 Box (i.e. “Like Extremely” and “Like Very Much”)

• Attribute Agreement (5-point scale)

  • Benchmark: 70% “Strongly agree”

• Performance vs. Expectations (3-point scale)

  • Benchmark: 70% “Better than expected”

  • Red Flag: 15% “not as good as expected”

• Forced-Preference (2-point scale)

  • Benchmark: 60% prefer your product to the comparison product

• Just-about-right (JAR) Product Diagnostics (3 or 5-point scales)

  • Benchmark: 70% “Just about right”

  • Red Flag: ≥15% on either side of the JAR scale

What about Purchase Intent...where’s the benchmark for that?

Certain measures cannot have a hard success criterion without context (e.g., volumetric projection w/minimum hurdle, etc.)

For example: 5-point Purchase Intent, probably our key measure in most branded marketing research studies could looks as follows.  Consider the following two concept scenarios:

In this example, Scenario 1 is the profile of what could be a strong niche business among a very small and vehemently loyal buyer-base with the majority of the population having no interest in ever purchasing.  On the other hand, Scenario 2 could be a large and mature mainstream brand with light interest among a very large and broad swath of the general population.  Both could be measuring viable brands and businesses.  Accordingly, it would not make sense to apply a strict research-based benchmark to Purchase Intent or other similar measures.  Instead, in the case of Purchase Intent, these scores are generally utilized in volumetric projections that could yield a better benchmark or minimum hurdle for action.

---