Repeat Shoppers Are Very Satisfied With Service

I always have a good laugh when I see how people use data in a way that frames something to be more than it really is. The title here is a made up scenario, but I can clearly see a retailer that sends out satisfaction surveys to people that shop at their store two or more times in a 60-day period. They then put out a report that shows that their customer satisfaction scores are 90%.

Of course, without understanding the selection bias, it sounds great. [This is not dramatically different than claiming outcomes that are simply regression to the mean which has been an issue for years across a lot of companies.]

In another example, a vendor talks about consumer reactions to their communications. They point out some great quotes that people say about their interactions. But, there’s a HUGE selection bias here. The people that make those comments are:

• People who are willing to interact with the technology (i.e., pick up the phone and listen to the message);
• People who spend time on the entire call and make it to the end of the call where there’s a survey; and
• People who are willing to participate in an optional survey about the communication.

Should we be surprised that these people really like the technology? NO!

It’s like asking people in a web survey whether they frequently go online.

But, that’s the beauty of sharing pieces of data. And, it’s proof that words do matter.