(A continuation of a discussion about the misuse of percentages in research)
What if I told you that if you shook my hand that you would have a 0.1% chance of catching a cough? (A fictitious statistic for the sake of this discussion). Would you? What if you would have a 0.1% chance of catching Ebola? How long would it take you to run as far from me a humanly possible?
Often, the odds of an incident are only half of the story. Severity is another part of the equation that we can’t overlook. This usually seems straight forward when we talk about issues like catching Ebola, tigers attacking their trainers, and other “black
swan” types of events. That’s the role of things like insurance. But what about the more mundane things that we measure?
Consider a recent satisfaction survey at a conference. In this survey, 20% of the attendees give the quality of the speakers low marks while 50% give the quality of food at the conference low marks. What issue needs to be addressed? At a glance, one could assume that putting resources behind improving the food is a sound strategic next step. But what if deeper customer research had been done and it is discovered that it is ten times more important to have quality speakers than to have quality food? What really needs to be “fixed” to improve the conference?
The proliferation of numerous web and print platforms for surveys has certainly democratized the role of research. Anyone can go online to generate and distribute free feedback surveys. But how often do you see these surveys asking about importance of the different options or asking to compare the preference of choice between two different types of attributes? The numerous free platforms are great at putting questions in front of people and putting them into attractive graphs and charts. But are you getting the whole truth from those numbers? Are you adequately informed to make the right decision? Are you sure that your business isn’t in danger of catching Ebola instead of the common cold?