Why Modern Product Benchmarking Should Listen To Real People

Research for product development is a funny thing. The metrics that look great in your marketing meetings may not tell the story of how your product actually performs in consumers' homes. So while your team is bringing out the champagne for beating competitors in brand awareness, your real users might be quietly switching to other products that better fit their daily lives.

Most companies still rely on traditional benchmarking methods that prioritize what's ''easy'' to measure and compare: market share, brand recognition, technical specifications. These surface-level comparisons miss the real story of how products perform in consumers' daily lives.

Yet the only real solution if you want to prevent churn is creating better products. And if you understand how your product performs against competitors in real homes, with real consumers, in real situations, you have a decent shot at outpacing other brands. 

That means it's time for benchmarking a la 2025. We'll explore why modern benchmarking needs to go beyond lab tests and market data to understand true product performance. 

You'll learn how leading brands are using real-world testing to gather authentic competitive insights, why traditional benchmarking methods often fail to predict market success, and how to set up benchmarking that actually helps you develop better products.

Why product benchmarking matters

Brands can benchmark all kinds of things—but product benchmarking is often overlooked. We'll look into the reason behind that in a second, but first we want to make it clear why you shouldn't skip it in your next round of product development:

• Surface-level metrics like brand awareness can mask deeper product issues. A product can be well-known and still lose to competitors in actual usage.
• The difference between lab success and real-world performance can make or break your product. A technically superior product might fail simply because it doesn't fit into consumers' daily routines.
• Your competitors' dissatisfied customers are your greatest source of product insights. Understanding why they're unhappy with current solutions helps you build something better.
• Product benchmarking de-risks innovation by revealing what actually drives consumer preference, not just what shows up in market research or focus groups.

If that isn't enough, IHUT (or in-home usage testing) testing for product benchmarking also gives you rich insights and anecdotes that you can directly apply to your marketing messaging, making it much more unique and relatable to your target audience. So why isn't everybody doing it?

The historic challenge of product benchmarking

There are plenty of reasons for brands to double down on product benchmarking. And while platforms like Highlight make it much more accessible to actually do, a lot of brands still are stuck in the mindset that in-home product testing for benchmarking is too complicated. 

Historically, the CPG industry has struggled to obtain and maintain reliably accurate and transparent benchmark data. Brands have had little to no transparency into what products their own offerings were being benchmarked against, or whether those benchmark products even successfully launched to market themselves.

This lack of transparency has made it difficult for brands to make informed decisions about product development and launch strategies. What's needed, and what is absolutely possible, is a fundamentally different approach to benchmarking—one that provides:

• Transparent data collection across all CPG sectors, including market leaders, consistent performers, disruptive brands, and even lower performers
• Standardized measures that include key metrics like purchase intent, value perception, and relevance
• Testing with consistent audiences that reflect both the general population and target consumers
• Clear understanding of benchmark products and their market performance

Before we reveal how you can set up your own product benchmarking process, let's take a closer look at how it works, and what you'll measure.

Product testing vs. product benchmarking

How is product testing any different from product benchmarking? And what difference do at home tests make?

Simply put, product tests simply test how a product performs. Product benchmarking compares these performance metrics to other competitor products. And IHUT product testing or benchmarking does this in the homes of real consumers, far away from squeaky clean lab environments. 

All of these research methods have value, but it's crucial to understand what they bring to the table and when to choose which. Here's how these methods compare:

Why at-home consumer tests make a difference

While you could run benchmarking research in a lab and get far more controlled results, there's a lot to be said about in-home testing for benchmarking. People's lives, routines, and habits all influence how they use a product, which affects how the product works. 

A product might have superior technical specifications, but if it's difficult to store, complicated to maintain, or doesn't play well with other products consumers use, it won't succeed in the market.

Take a coffee maker. Traditional benchmarking might focus on brewing taste, or time. But real-world testing could reveal that consumers actually care more about how it fits into their morning routine, how easy it is to clean, or whether it's quiet enough not to wake their family. These insights are hard to get in lab testing or market share analysis.

Some of the most crucial human elements and metrics that impact product success are listed below:

Setting up effective product benchmarking

So now you've decided that real-world product benchmarking needs to be part of your development process. But getting your products into the right homes, gathering meaningful competitive data, and ensuring structured feedback isn't as simple as sending out samples and hoping for the best. 

The success of benchmarking studies depends on having a strategic approach that addresses every possible hurdle, from participant selection to data collection methodology. 

Here's how to set up a benchmarking program that delivers actionable insights while protecting both your products and your investment.

1. Identifying the right comparison targets

Before you send out any products or even select any research participants, know what you'll be benchmarking against. This is crucial for your research design, but also to filter out the people you want to send your products to. You could go for:

• Direct competitors in your specific category
• Category leaders who set consumer expectations
• Multiple products across different price points
• Products that solve similar problems in different ways

Whichever one you choose, make sure it is clearly defined and that the market research questions you ask make for relevant comparison between you and your competitor.

2. Finding the right users for benchmarking

Gathering insights from the right people is one of the main priorities in product benchmarking. Traditional demographic matching isn't going to cut it. 

The most valuable feedback often comes from people who regularly use competitor products, and have done so for a long time. They understand category nuances, have developed usage habits, and can articulate specific advantages and disadvantages that matter in daily use.

But finding these users isn't just about sending out surveys to people who've indicated they know your competitor's products. At the very least, you need to:

• Confirm that they've actually bought the product (through purchase receipts and/or photos)
• Verify they've used the product for an extended period (via video or photo evidence)
• Check they fit the right usage patterns (pre-survey questions)

That's a lot of pre-research to do, but at Highlight we do it for you. We're picky, meaning you get the most relevant and accurate research data. Here's how we do it. 

3. Create meaningful comparison metrics

The main goal here is to balance quantitative data with qualitative insights. Modern benchmarking requires both hard data and human insight to paint a complete picture of product performance. Quantitative metrics tell us what's happening, but qualitative insights tell us why—and in product development, the “why” often makes all the difference.

When you combine both types of data, patterns emerge that neither could reveal alone. For example, quantitative data might show that users of a competitor's product use it more frequently than yours. But only qualitative insights can reveal that it's because their packaging makes it easier to store in small bathrooms, or because their formula works better with hard water in certain regions.

• Performance metrics from actual usage
• User experience narratives
• Preference patterns over time
• Contextual influences on product performance
• Unexpected benefits and drawbacks
  
  
  

4. Turning your insights into action points 

The research results are just the beginning. Here's how to translate benchmark data into actual improvements:

• Product Development
• Identify feature gaps that matter to consumers
• Understand usage patterns that should influence design
• Discover unexpected pain points to address
• Find opportunities for meaningful innovation
• Marketing and Positioning
• Develop messaging based on real competitive advantages
• Understand true purchase drivers
• Create authentic value propositions
• Support claims with real user experiences
• Customer Experience
• Improve packaging based on usage patterns
• Enhance instructions and support materials
• Develop better user onboarding
• Address common usage challenges
  
  

Trust real consumers for your product research

The most valuable benchmarking insights don't come from spreadsheets or labs—they come from the homes of people who use your competitors' products every day.

Want to know how to win in the moments that matter to consumers? Transform your approach to product benchmarking with real-world testing.