Market Research

Having recently talked up working in a call centre allow me to now talk it back down. Well, not really but many of my colleagues + facebook friends often complain about the job, and I hope they are complaining about the job itself.

My employers a good one as far as market research goes, we have opportunities to give feedback direct from the coalface to the clients and researchers that write our questionaires and so fourth and frankly I'm not privvy to what goes on behind the curtain so I'm not really qualified to comment.

Suffice to say, I'm not really convinced of the merits of the whole industry of market research myself. I'm qualified in marketing and plan to spend vast parts of my future not using these qualifications at all. Not because I hate marketing, I hate the practice of marketing rather than the exciting soft science that it is.

My experience of marketing is that people who are 'professional' marketers are most often brought in to be paid to agree with whatever manager or exec thinks is a good idea as to how their advertising campaign should run. Typically this results in some engineering marvel that is impenetrable to the general public (like ABS braking in the 90's) featuring front and center in your advert and being referred to in lingo with no demonstrations of its actual benefits.

Or as I tried to do in my limited capacity - you point out to the exec or manager attempting to employ your professional endorsement that their idea is in fact terrible they roll over you.

You'd think market research would thus be impenetrable, the survey is vast, statistically significant with questions written to guaruntee data is comparable. So surely this would thwart any executive hoping to blow some money paying somebody to agree with them. The data doesn't lie now does it?

Well this is just hearsay, but I've heard of a manager at an unnamed Market Research firm with international operations recieve a call from clients requesting that they 'change the results'.

Now at my company if I got busted falsifying data by the manager I'd be fired, at the least suspended or sent home. But here the client is telling management to simply change the results of the research because they don't like what the research suggests. In effect making all that industry compliance and expense entirely unneccessary. A complete fucking waste of time.

But that's not even the reason I question the relevance of the entire industry. Statistical studies like the census and some of the surveys I work on are potentially legit estimating measures but market research broadly speaking is designed to overcome 'qualitative' issues, not by and large 'quantitative' ones.

That is, trying to find out how people feel, what they like, what they want etc. Changing or modifying a product is risky business. So naturally those assuming the risk want to hedge their bets. So they go interview the existing fans, the existing non fans see if they can find any consensus on improvement and then implement those changes. (to oversimplify the process greatly).

What has me doubting all this is Tool. Specifically Tool fans.

Tool Fans all have something in common, they like Tool. They fork out actual money for tool albums and tool concert tickets. But after that the common ground starts shrinking.

Here is my question? Could market research produce tool? That is in a world where there was no Tool, by interviewing the people that constitute tools fanbase would they be able to identify what was lacking in the music industry that they crave?

To me, this is as unlikely as a hurricane whipping through junkyard and assembling a fully functional Boeing 747 - and not actually too dissimilar a process.

You may not know tool, you may not know toolfans but you may be able to relate to my experience. My experience is that whilst all toolfans like tool, almost all toolfans don't like toolfans. Maybe its more accurate to say they don't really relate to eachother, they are not a group of people that would naturally congregate in large numbers without the central point of tool.

Even if you reduce toolfans to the group of 6 or so peeps I go to their concerts with, I don't think a study even focus groups or whatever would produce a description of Tool when it came to collating our common desires for music.

I feel like I'm poorly articulating this, so let me talk about diversification and aggregate average expertise. Diversification is most commonly associated with investment, it means spreading your assets as widely as possible to counteract market volatility. This reduces your risk. In other words, you have a company ABC and company XYZ, when ABC moves up, XYZ generally moves down and vice versa. By having the two assets in your portfolio it means that any fall in value of XYZ is offset completely by a gain in ABC - this lay description is called 'portfolio theory' where the ideal is to achieve a capital gain of zero. This is attractive because it protects your principal (the cash you invested in the first place and would be returned to you if you sold out) but you would still earn all your dividends so your net investment is positive.

Warren Buffet says 'diversification is the best way to ensure you don't make any money at all' which is true, but risk = volatility therefore positive and negative deviations most people are more afraid of losing than they are of gaining.

Now aggregate expertise. I am an expert on South Dakotan Ornithology, you are an expert in WAM! cover bands. We are both world class experts, but completely ignorant of eachothers fields. If somebody was to interview one of us, they would achieve 100% good feedback in one field, and 100% complete garbage feedback in the other. If a researcher interviewed both of us they would get 50% valid feedback in both fields.

Now generally of all the vague 'subjects' or 'fields' one can have expertise in, chances are we are going to be complete dunces in most of them. The fields we are not expert in are so numerous as to dwarf the fields we are. Our average expertise is something incredibly close to 0.

Thus the sample of people in this case is like the diversification in an investment portfolio. The more people you add the closer the quality of the feedback will approach 0. Although it needs to be pointed out that it depends on how specific the field of interest is.

But for something like creating a band that will sell big to a dedicated subculture, I feel it would be rather like dumping rubbish infront of hurricanes in the hope that something resembling a jet will be assembled.

Fans of things will probably differ in their reasons for why they like things, furthermore I believe there is a substantial body of research suggesting that people don't even know what it is they do like. Perhaps most famously the 'let your taste decide' Pepsico campaign, which measured people's subjective experience of cola sweeteners but not their actual preferences. Their actual preferences were for Coca-cola.

I thus have little faith in market research's role in any creative process. As my old marketing lecturer pointed out even with focus groups 'you don't know if 1 in 100 person's opinion is actually valid.' it is furthermore a return to subjectivity to actually sift through and get the 'really great' feedback out.

My experience is that the most valid opinion usually belongs to the person actually responsible for creating something. Focus groups, test audiences etc I feel are of little benefit. I also suspect it is this aggregating effect that makes User Generated Content so overhyped.