Author: Nadine Touma Nadine Touma is a consultant in Insights, Strategy & Innovation based in Dubai. In her blog, Let’s talk business, she shares her 20 years of experience in the corporate world and 5 as an independent consultant with real-life stories, simple advice, and a no-nonsense approach that characterizes her work Original Post
As per the famous motto “garbage in, garbage out”, the quality of insights is inherently linked to the quality of data, and the way said data is collected (not only, as we will see in Part 5 to 7, far from it actually, but bear with me here). In Part 1 and 2, I illustrated these points with third-party data, in this article I would like to focus on primary consumer data…
First of all, allow me to address the detractors, the ones who always point out the times when the “consumer was wrong”. As much as I am a fan of consumer research, I remain very clear on the fact that, indeed, it is not gospel truth…
The point of consumer research is not to be “right” or “wrong” anyway. It is to identify patterns, decode trends, understand triggers, motivations, behaviors so that businesses can perform better and innovate.
So whether the percentage of people who, say, shower every day, is actually 78% and not 87% (those are totally made up numbers for the sake of the example), it is, granted, not totally accurate, but not the end of the world either (albeit a bit disgusting for those who don’t shower) and the conclusion quite clear: we can continue selling shower gels but how to increase usage?
More seriously now, there is a lot to say about consumer research and I will most certainly address it in several other posts, however, I would like to point out here some common mistakes that could lead you, indeed, to not “trust” consumers.
When the questions are too complicated, not intuitive, you cannot and should not trust the answer.
Years ago I worked with a global research company, reference in its sector, and was absolutely baffled by the questionnaire they had designed… Among which, question: “How much do you spend, per year, on luxury? and by luxury I mean: perfumes and cosmetics, apparel, leather goods, shoes, accessories, watches and jewelry”… Errrr, excuse me, do you really think consumers have any clue? 1) The question refers to many categories all combined together 2) Those product categories have nothing to do with each other in terms of spending patterns and purchase habits 3) AND you are asking them to mentally cumulate all that over a year… Kindly ditch the question. Thank you.
It is well known that you cannot rely on answers given on spending (or income) because it is recalled, it is claimed and the subject of money is never easy to approach. However, even if the absolute value will NOT be accurate what is interesting is to understand the purchase habits and ask questions that are easier to answer, more intuitive, eg: “How many times do you go to purchase (insert category)”? “And how much do you spend on average?” (suggest brackets). At least people here can focus on the particularity of each category and not try to comprehend the intellectual notion of “luxury”.
Chances are the frequency will be relatively accurate but the derived yearly spend won’t, because they don’t necessarily remember average basket accurately (who does?) and because they can subconsciously exaggerate (at least in the Middle East). However differences between one category and the other should be relatively accurate, indexes/variances by socio-demographics should be relatively accurate. So the fact that the absolute value is “wrong” doesn’t prevent you from drawing a lot of interesting insights out of these questions… as long as you ask them with a minimum of common sense, that is.
Do not bore respondents with useless questions that will bring you no strategic or operational insights anyway.
At the very beginning of my career as Head of Market and Consumer Intelligence for the Chalhoub group, I needed to establish a baseline for luxury consumers’ usage and attitudes (U&A) for our categories, namely perfumes & cosmetics, and fashion & accessories. Back then, I trusted quite blindly global agencies with an established local presence, they are the experts are they not? The first draft questionnaire I received on fashion and accessories read something like this… “How many pants to you own?” “How many dresses do you own?” “How many tops do you own?” How many bags do you own?”… you get the gist of it.
So first of all, treating luxury categories as if they were detergents doesn’t seem like a bright idea. Second, in reference to my above example, do you really think they have any clue about the answer? (do you know how many pants you own?). Third, I couldn’t care less about how many they OWN, what I would like to understand is how they PURCHASE. Fourth, imagine how motivated they will be after those riveting questions and the effect it will have on the accuracy of the remaining answers. And most importantly, fifth, what kind of insights can I draw from those mind-numbing questions? None, zilch, nada.
No, in this example, what I need to understand is purchasing patterns, triggers, motivations, attitudes, and… wait for it… emotional associations! Luxury is a category inherently linked to dream, status, emotions, feelings!
Side note: it is from this first questionnaire that we learned that filtering by income was not a good idea, contrary to the common practice in the West. 1) People tend to exaggerate (Welcome to the Middle East!) 2) Women don’t necessarily have an idea of how much the husband/household earns (Welcome to the Middle East!) 3) Most importantly there is a sizable portion of youth that doesn’t “earn” very much but spends it all on luxury items… Filtering by income means missing out on a very important target group.
Don’t ask quantitative questions in a qualitative setting.
I cannot count the times when I had to fight with a client who wanted to include a “and how much would you spend in this new store?” question in the middle of qualitative concept testing. Again, what type of insights would you draw? Think about it. You have 4 focus groups, of 8 people each, with totally different socio-demographic profiles. At best, half will answer the question. Perhaps a handful would say Dhs100 and a few Dhs500. So what? Are you going to conclude you can target an average basket of Dhs300, based on 15 answers? I hope not.
Verbatims are to “qual” what statistics are to “quant”. Focus groups are not here to tell you how many people think something but what they think and how they formulate it. “Quant” will then give you numbers to assess the weight of the what.
Consumers don’t know what they want, they know what they DON’T want, what they like and dislike. It is up to you to come up with novel solutions based on consumer insights. Not the other way around.
In the famous words of Henry Ford, “If I had asked consumers what they wanted, they would have answered faster horses…”
20 years ago (gasp!), I worked as a Senior Consultant at, what was back then, a start-up called Integration (now Brand Experience Group – BXG), which devised a pioneering method to optimize all elements of the marketing mix and calculate the ever so evasive Marketing ROI. Mind you, back then you only had some techniques to measure above-the-line marketing activities and there was no way to evaluate with “a common currency” things as diverse as advertising, POS activities, recommendations, word of mouth, etc.
The method, called MCA® (Market ContactAudit®) was extremely simple and yet incredibly effective. Consumers were asked what activities they didn’t find informational, the ones they didn’t find attractive, to then go on to a ranking and an association between competitive brands and said activities. The first three items were computed to form a Contact Clout Factor (weight/importance of activity) and, together with brand associations, you could then come up with many analyses around Brand Experience, differentiation, marketing activities on the rise, and ROI.
Simple. Intuitive. Maximized chances at being accurate. Over time, it was demonstrated that the Brand Experiences calculated through that method were not only correlated at 80% to Market Shares but also constituted a leading indicator, ie predictive, of the evolution of Market Shares… All with 4 questions. Drop mic.
More generally, don’t ask consumers what the “ideal store / ideal service/ideal whatever ” is(they will ask you for a nursery in the middle of a Vuitton store… true story). Ask them what they currently like, dislike, ask them about how certain things make them feel, based on very concrete examples, be it of actual stores or concept boards. Full stop. How you interpret that is the job of professionals. Solutions/concepts can be tested again and again until you come up with your star solution.
Beware your sample and the conclusions you are drawing from it!
Just like you should judge the quality of your questions (or their irrelevance) by the type of insights they will provide you at the end, whether operational or strategic, you should establish your sample the same way.
Contrarily to common belief, your sample does not necessarily need to reflect the population of the country you are analyzing, but your target groups (primary and secondary). Unless you have no idea who that is.
For example: when analyzing luxury consumers, you will of course select respondents according to particular purchase habits (filtering by brands purchased, frequency, or spend. Not income, for Middle Eastern consumers, as explained above). But more than this, you have to focus on the cities where you have a presence or are aiming at having a presence.
Why? Take Saudi Arabia. Population 35M. Riyadh + Jeddah about 11M. Unless you have a presence throughoutSaudi Arabia, there is not only any need to have respondents all overthe kingdom but it will actually give you insights that are not relevant to your business or industry. It seems pretty obvious, I know. And yet, in my first attempt at consumer research in the Gulf, the agency I had selected (big name) set up a sample over the total population, that I validated due to my lack of experience in the region, to soon learn my lesson as luxury brands were only in the 2 above-mentioned cities (plus Khobar but that is a detail).
Knowing that consumer research costs a lot (particularly when you have to carry it face to face), it is a good idea not to waste money on a sizable unusable portion of a sample that will dilute your total results on top of it. What is best is to build the sample from the bottom up. Identifying the smallest cell you want to read individually, say, by city / by gender / by age, fix it a size of 50 or more (smallest size statistically representative) and then build your blocks and come up with the number you need with a distribution that is relevant to your industry and granular enough for future analysis.
There are many other examples and learnings I could share with you on consumer research. I will leave them for other articles. But I will conclude with a simple remark that seems to be recurrent in all my posts…