0

Market Research Quality Definition – Criterion Validity vs. Construct Validity

The differences between academic and commercial research

Market Research Quality Definition – Criterion Validity vs. Construct Validity

Image courtesy of [KROMKRATHOG] / FreeDigitalPhotos.net 

I am currently at an academic conference on mobile research in Cyprus, a WebDataNet event. I am a keynote speaker and my role is to share with the delegates the commercial market research picture.I really enjoy mixing with the academic world, and I am intrigued and fascinated by the differences between the academic and commercial worlds. This post looks at some of the key differences that I have noticed.

Timelines
In the academic world, timelines are usually longer than in market research. For example, an ethnographic project might be planned for 8 months, in the field for 4 months, and spend 12 months being analysed and written up. A commercial ‘ethnography’ might spend 4 weeks in design and set-up, the fieldwork might be wrapped up in 2 weeks, and the analysis and ‘write up’ conducted in 2 weeks.

In many ways the differences in the timelines result from differences in the motivation for doing a research project. Commercial market research is often conducted to answer a specific business question, which means the research has to be conducted within the timeline required by the business question – which is typically rapid. Academic research is typically conducted to advance the body of knowledge, which means there is often not a specific time constraint. However, there is a need to establish what is already known (the literature review) and a need to spend time creating a write up that embeds the new learning in the wider canon of knowledge.

The balance between preparation, action, analysing, and writing up

In the commercial world the answer is the point of the study; the method, providing it is acceptable, is less relevant.

In an academic study, the value of the specific answer is sometimes almost the least important feature of the project. For example, a commercial project looking at five possible ads for a new soft drink would seek to find the winner. An academic project would normally find that sort of result too specific (i.e. not an addition to the canon of knowledge). An academic project might be more interested in questions such as, what is the relationship between different formats of ad and the way they are evaluated, or the extent to which short-term and long-term effects can be identified. Indeed, in academic project the brands and the specific ads tested will often be obscured, because the study is about the method and the generalizable findings, not (usually) about which ad did best.

The definition of quality
Academic and market researchers have a hierarchy of types of validity but the hierarchy is not the same. Market researchers tend to value Criterion validity (does the measure correlate with or predict something of interest) as their ‘best’ measure.

By contrast, the academic world tends to prioritise Construct Validity, which relates to how well new findings relate to an accepted theory of how things work. This again probably relates to the specificity of the objectives. Market researchers need something that works well enough to solve a particular business problem. The academic is seeking to build knowledge and to connect that research to a wider framework.

The difference in samples
Most market research is conducted with a sample drawn from the target population and usually the sample is constructed to be similar to the target population in terms of simple variables such as age and gender – although it usually falls well short of being a random probability sample. By contrast, a large proportion of academic research appears to be conducted with convenience samples, often students.