You know you need data to build the best possible product. But there’s still so much more to decide.
Should we purchase retail measurement data?
Should we invest in lab testing?
What kind of panel should we test with to reflect our target audience?
The decision to conduct product testing is step one. Settling on the most efficient and effective methodology is step two–and that can be a lot harder to determine.
No one product testing method is one-size-fits-all. That’s why we’ve created this in-depth guide to provide all the considerations you should make in order to make the best decision for you, your teams, and your product.
Beginning with a little desktop research can go a long way.
Tech-based product testing companies may be listed on trusted review websites like G2, Capterra, or Trustpilot.
Physical product testing is a more specialized niche within the larger product testing space, so if you’re specifically seeking companies to help you conduct testing of physical products, look to expert associations like Insights Association, ESOMAR, Greenbook, or Quirk’s. These associations are dedicated to advancing consumer insights as an industry and often provide services to vet potential product testing suppliers.
Don’t forget to leverage your professional network! Reach out to colleagues or former colleagues to ask for word-of-mouth recommendations, or post on LinkedIn asking for advice from those with product testing experience.
Building a list of product testing companies with solid user reviews, endorsements from industry associations, and referrals from your network is an important starting point. But you’ll need to get more specific to make the right decision for your needs.
With so many different methodologies for testing products, it’s only natural that there are about a million different kinds of product testing companies. A product testing company may specialize in a particular methodology, a particular industry or vertical, testing with particular audiences, or any combination of the above.
The most common product testing companies can be bucketed into the following categories:
Digital product testing companies primarily serve digital products like software and mobile applications (apps), but they might also test digital versions of a physical product.
If you have a physical product, a digital product testing company may not be the right fit for you, except for in the case of a digital version of your physical product. Take, for example, the case of concept testing. You may not have a product yet, but you still want consumer feedback to decide on an initial direction.
This is especially common in apparel testing for fashion products. Maybe you’re developing a new spring line of clothing, but you need to know what colors and patterns appeal to consumers the most. In this case, a digital product testing company could be a great option to show product testers images and ask them to choose their favorite(s).
CLT is a tempting option for many companies looking to conduct product testing because it feels relatively cost-efficient. By recruiting product testers to come to a specific location, you can observe while they unbox the product, use the product, and react to the product.
In-person focus groups are another commonly used methodology within CLT. Nowadays, you’ll find more and more product testing companies offering online focus groups, too.
IHUT is a great alternative to CLT. Historically, IHUT has been challenging thanks to complicated logistics and the limits of data collection, but technology and supply chain advancements have made modern IHUT a much more accessible and efficient option for product testing.
The advantage of IHUT over CLT for product testing is the ability to measure consumer feedback to a product in the natural environment of their own home. If you wouldn’t bake a cake in an office, why would you ask your product testers to do so at a CLT? With the right IHUT product testing company, you can put physical products in the hands of product testers quickly, and receive their feedback easily.
Brands might choose professional lab testing for a number of reasons. Let’s discuss two common use cases for professional lab testing: health & safety, and professional sensory assessment.
Many brands may choose to test their products with a professional lab to ensure the health and safety of their products. If you’re a children’s product, for example, who works with third party manufacturing partners, you may want to enlist a professional lab testing company to ensure there are no dangerous levels of lead or other potentially harmful chemicals in your product.
Professional sensory testing is another lab test use case. While a new wine, for example, may benefit from real consumer reactions in the context of CLT or IHUT, wines are often tested with professional sensory experts like sommeliers to make sure their product is truly hitting the mark.
If you’re a small brand, you’re probably also very ambitious. In the CPG and retail worlds, some of the most exciting innovation comes from the smallest companies. But the price of being this ambitious is that many small brands try to boil the ocean. And the truth is, when you’re still small and growing, a little goes a long way.
If you’re a small company looking to test your products with the right partner, you need to consider the following:
Small brands need an option that allows them to move at lightning speed, make a big impact with a few key data points, and they need to do all that within a tight budget. Often that means product testing companies that use technology to offer speed and accessibility to impactful insights.
Another consideration for smaller brands is product testing expertise. While the founder(s) or other founding members of the team may have some product testing knowledge, they are busy, and they should work with product testing companies that have reliable expertise – either on staff or productized, preferably both! – to run product tests on their behalf while they focus on running their companies.
For enterprise brands with bigger budgets, they have the luxury of choosing to work with similarly large product testing companies, such as NielsenIQ.
For many enterprise companies, large product testing companies are a great option for the white glove service they provide. With whole teams of human researchers as partners and service-level agreements to ensure thorough and responsive support, many internal research and development teams choose to work with such large product testing companies.
Other enterprise brands may actually choose to work with smaller product testing companies for two key reasons:
Historically, large product testing companies simply don’t have the capacity to move at more digitally-native small startups. Nowadays, as product innovation cycles shrink from 18 months to 12 months, and from 12 months to six or even three, working with smaller product testing companies may be an attractive option.
While enterprise budgets also tend to be larger, innovation teams still have to justify costs to their superiors. If a quick and relatively inexpensive product test can provide data feedback enterprise teams need to refine an in-market product or validate a product before launch, smaller product testing companies can be a great option to meet those needs.
Generally speaking, with a physical product built with the intention for home usage, in-home product testing, or IHUT, is a crucial product testing step, even if the methodology may differ.
Within CPG, retail and beyond, everyone from yogurt to mattresses can benefit from in-home usage testing to understand the real, organic customer experience and make adjustments to ensure the product is living up to consumer expectations and earning greater “share of cart.”
This includes product categories such as:
And more–read on to dive deep into the details for product testing each category.
Food and beverage brands face a unique challenge with product testing.
Often, especially in larger companies, multiple teams are involved in product development and launch. This approach can sometimes lead to siloes or even conflict of interest.
For example, a research and development team may be particularly concerned with sensory testing, while the brand team preparing the launch campaign for the product might be more concerned that the product hits the mark in terms of consumer trends.
Imagine a new, better-for-you soda is in your innovation pipeline. The R&D team needs to know, does the presence of 50% less sugar impact sweetness levels so much as to alienate customers? But at the same time, the brand marketing team has totally different questions: Does the presence of “50% less sugar” claim on the packaging impact willingness to purchase? Is there another way to phrase the sugar content claim to make it more compelling to consumers? Or should we advertise the use of healthier monkfruit sweeteners in the soda instead?
Some of these questions can be answered via digital product testing, but others require the consumers to use the physical product. And while consumers could interact with the physical product in a CLT, the data might be more reliable when gathered in the organic context of the consumers’ daily routines at home via IHUT.
At Highlight, we recommend consolidating your approach to be more efficient. When you use IHUT to gather sensory and consumer insights in one product test, every team can get the data they need to succeed.
Like food and beverage, beauty and personal care product testing is often interested in two sides of the same insights coin: the sensory experience and whether the product is “on-trend.”
Many beauty and personal care brands also have to consider efficacy and product testing methodologies like claim substantiation. If your anti-aging serum claims to reduce wrinkles, you better have the product testing to back that claim up–both for consumer trust and legal reasons.
For beauty and personal care brands that need rigorous legal claims substantiation, professional, in-lab product testing is often a necessary step to mitigate risk and make sure you are covering your bases.
For many other beauty and personal care brands, your approach to claims substantiation may be different. For example, a claim stating that your supplement “is proven to” increase energy levels is very different from a claim stating that “9 out of 10 users reported higher energy levels.”
In other words, where consumer perception is the crux of the claim, consumer testing, usually via in-home usage testing, is the best way to go.
IHUT often makes the most sense for beauty and personal care products because of the nature of these products. For those who regularly use beauty and personal care products, they are often a part of regular, everyday routines, often in intimate or private moments of a consumers’ day. These kinds of products also tend to be used regularly over time. So while sensory questions could be answered in the moment (such as “How appealing do you find the fragrance of this lotion?”), many questions around efficacy and consumer satisfaction require protracted product use to accurately assess. That’s why so many beauty and personal care products conduct longitudinal studies to get the data they need to build better products.
Product testing for apparel and footwear brands present yet another niche use case.
Like beauty and personal care, apparel and footwear products require testing in a consumer’s natural, everyday environment and routine to understand how the product holds up, which is why IHUT is the standard recommendation. And also like beauty and personal care products, apparel and footwear require longitudinal testing to understand key product characteristics like durability and comfort.
Another factor to consider in apparel and footwear is the cost per unit in comparison to CPG products. Because it can be much more expensive to distribute something like a pair of shoes to product testers, these brands are often restricted to smaller sample sizes.
To combat this restriction, it’s important to “go deep” with your apparel and footwear product testing by incorporating more qualitative methodologies, such as diary entries and video responses.
Watch this example from a Hanes apparel test: Open-ended video response mines insights you might otherwise miss in quantitative surveys
You can learn more about apparel testing in our dedicated guide: 11 Considerations for Apparel Product Testing.
Products built for kids or pets require a nuanced approach to product testing. For one, while you may ostensibly search for feedback from the child or pet actually using the product, the truth of the matter is that you’re gathering feedback from the guardian’s perception of that child or pet’s enjoyment of the product, as they are the individual making the purchase decision.
With children’s products, that can be challenging. Legal parameters require guardian consent, and questions should address both the child’s enjoyment of, for example, the flavor of an apple sauce, and how the parent feels about, for example, the sugar content advertised on pack.
Likewise with pet and pet care, product testing needs to catch several dimensions of the product experience. If you’re testing a kitty litter, for example, the consumer needs to enjoy the sensory experience of the deodorizer fragrance, but the cat also needs to consistently use it without rejection or other signal that the owner may interpret as dissatisfaction.
In either case, IHUT usually presents the best option for assessing the quality of the product experience. (Can you imagine bringing 50 cats into an office for a CLT?)
For more information on how IHUT can be used for these special product testing scenarios, we recommend these case studies:
Tree Top Applesauce tests their line with 75 pairs of parents and children
National cat litter brand tests their new SKU with 7-day IHUT
The kind of data you can expect from a product testing company depends on what kind of questions you ask, and the best product testing companies will guide you on the best questions to ask to satisfy your business need(s).
Let’s look at Highlight’s methodologies by way of example:
Let’s take a common use case of Prototype Sensory Evaluation. In this kind of product testing, you can expect data around:
Data will also vary by incorporation of quantitative and qualitative data collection methodologies. While a quantitative sensory evaluation might use JAR scales (“Just About Right”) to assess sweetness levels, a qualitative approach might help serve the needs of the brand marketing team.
In one example of an Agile Co-Creation approach for ideation, a coconut water brand asked product testers to write “love letters” or “breakup letters” to the product. While it might sound silly on paper, asking consumers to choose their own words in this qualitative approach captures sentiment analysis, reveals patterns, and uncovers words or phrases that might be the perfect addition to their advertising efforts.
Whether you are testing for sensory or consumer insights (or both), using quantitative methodologies or qualitative, the best product testing companies will also incorporate questions that produce data you need not just for product adjustments and optimized marketing efforts, but also for business goals like retailer sell-in. For example, you should expect data points like:
This kind of data around purchase intent is key to winning more distribution for your product.
TL;DR: The kind of data your product testing company offers should depend entirely on your business goals, and it’s important to be clear on what those are before engaging a product testing company. The best product testing companies will work with you to extract data insights that satisfy those goals.
When considering a product testing company, you’ll want to come prepared with plenty of questions to ensure the right fit. Here are just a few ideas to get you started.
Understanding who will test your product(s) is key to ensuring the data you collect is as accurate as possible. You could get negative feedback from one group of testers, only to realize these folks don’t reflect your core consumers at all.
Make sure to ask this question to determine whether a product testing company has the proper demographic or psychographic reach you need to test with your core consumers.
One aspect of product testing that’s often overlooked is not just the selection of product testers, but the maintenance of an engaged community of product testers. You want to avoid product testers who are overloaded with requests. Ideally product testers should be engaged no more than once a month and never on multiple tests at once.
You also want to make sure that your product testers have the support they need to complete the study as designed. So be sure to ask questions around product tester support, what happens when they have questions, what happens when they have a problem with the technology or shipping, etc. to ensure that the product testing is run smoothly.
When it comes to physical product testing, it’s no easy feat to get products from the manufacturer, to a central warehouse for repackaging, and then into the hands of testers.
Supply chain management is crucial, so you want to make sure you are working with a team of professional supply chain professionals. You’ll want to ask questions around how they ensure the fidelity of your product is preserved for shipment (for example, if your product needs refrigerating or freezing), how your products are blinded, and how quickly shipments can be processed.
CLT encounters the challenge of product testing in unnatural environments, so when working with a CLT product testing company, it’s crucial to ask what kind of measures they are taking to conduct product testing in an environment that recreates a consumers’ organic routines as closely as possible.
If you’re paying the premium for professional lab testing services, your product testing company better be accredited by the relevant regulatory boards and associations. For example, if you are working with professional sensory testers, are they members of the Society of Sensory Professionals? What kind of degrees do the testers hold? Make sure you do your homework to make sure they’re applying the rigor that a professional product testing company should.
While a product testing company doesn’t necessarily have to have worked with the biggest names in the industry to be “legit,” it helps if they’ve worked with products or brands like yours in the past that demonstrate they can do the same for you. An experienced product testing company will have case studies to showcase past successes.
Transparency is important in any business transaction or partnership, but it’s especially important with product testing. Your audience of potential product testers are a crucial input to getting the data insights you need, so your product testing company should be able to answer questions about their panel clearly and quickly.
When it comes to IHUT, logistics and shipping can become complicated. But the best IHUT product testing companies work extremely closely and maintain a good relationship with their logistics partners so that they’re able to give transparent, reliable information on timelines and make the process as predictable as possible for you.
If you’re working with a product testing company that doesn’t have to manage the shipment of physical products and they still can’t give you clear timelines, that’s an even bigger red flag. You should know from the beginning a reasonable timeframe for your research.
The best product testing companies are staffed by veterans of the product testing industry. Whether their specialty is focus groups or sensory testing, they should be fluent in the language and methodology that it takes to get the data you need and provide you whatever level of guidance you need.
Highlight began as a solution to a painful problem that Dana Kim, Highlight’s founder, felt acutely as a product researcher herself. She worked with brands like Coca Cola, Nike, and Starbucks to collect the product insights their teams needed to fuel innovation.
But in a world where businesses have to move faster than ever before, she was blown away by how innovation was slowed down by collecting product insights. As she wrangled finding respondents to give feedback, shipping products, collecting data, and analyzing data, she realized that critical decisions were on hold for these insights which were taking months on end to materialize.
And, when Dana finally got the data, it did not meet the quality bar she set. Research was often conducted in unnatural, sterile offices clouding insights with inauthentic experiences of the product, the quality of panels was often suspect, and the methodology recommended by vendors was unsound.
That’s why Dana founded Highlight: to be a different kind of product testing company. To be the product testing company the industry deserves.
In practice, that means: