Use Semantic Differential Scales to Cut Survey Wordiness (Full Transcript)

[00:00:03] Speaker 1: Hey friends, Katherine here from Research Rockstar. Thanks for joining me here today. So today I'd like to talk about questionnaire design. And you know, one of the issues we often have in professional questionnaire design these days is making sure that we are not wordy. We want to be as concise as possible. And there are two big reasons why we're concerned about wordiness in questionnaires these days. One reason that we're concerned about wordiness is because we've learned a lot from the field of behavioral economics that it's really easy to unintentionally frame or prime something in a way that can bias responses. So we want to use as few words as possible so that we mitigate that risk. But there's another more practical issue that many of us deal with, which is many survey respondents now are taking our surveys from mobile devices, and that usually means smartphones. And even though many smartphones have gigantic screens these days, it's still a concern. We want to make sure that we are as concise as possible in our words in the questionnaire so that it will look really good on the screen. After all, some online survey platforms do a really great job of responsive design and giving us ways of graphically laying things out so they look attractive in a mobile device. But if it's too wordy, well, there's only so much that the device that the platform can do for us. So what I want to do first is start off by talking about the types of scales we often do see, perhaps too often, in even professional questionnaires. And what I'm referring to is the ubiquitous five-point rating scale. We see five-point rating scales all over the place, and we often will talk about our use of Likert scales. Now, of course, as a tangent, we could refer to them as Likert scales or Likert scales. To be academically precise, it should be Likert because the gentleman for whom the scale was named, his last name was Likert, and that's how it was pronounced. But many market researchers have learned from reading and from textbooks and have concluded that or, you know, assumed sort of that the scale was Likert, so a lot of people use that language. But in any case, you know, a typical Likert scale kind of thing would be, how would you describe your experience installing this software? And we give people a scale ranging from very difficult to very easy, right? And if it's a five-point scale, that midpoint might be a neutral. So it wasn't difficult, it wasn't easy. We're giving our participants a chance to indicate if they really just felt neutral about it. Now, of course, these types of scales are often presented as five- or seven-point scales, but some researchers do prefer even scales as well. And in some cases, researchers will even prefer longer scales like a 10- or 11-point version. But in any case, rating scales are clearly very common to us in professional research. But if we think about the wording of the questions and how it appears in a questionnaire, it does create a fair amount of wording in many questionnaires. How would you describe your experience installing this software? It's a fair amount of words. Or how likely are you to purchase a new car in the next 12 months? Fair amount of words. But let's look at what those words are and if we think that they might be biasing. So if I ask somebody, how likely are you to purchase a new car in the next 12 months? Well, some people would say that simply framing it as likely almost implies that the participant should have some likelihood. And clearly, that may not be the case. Some people may have absolutely no likelihood to purchase a car in the next 12 months. And so what some researchers will do is they'll reword that. Instead of saying how likely are you to purchase something, they'll say something like, to what extent are you either likely or unlikely to purchase a new car in the next 12 months? Or they might say something like, what is the likelihood that you will purchase a new car in the next 12 months? And that certainly does help to mitigate some of the ways in which the word likely could be leading when it's in the actual wording of the question. However, even if we are careful about making sure that there's nothing biasing in the wording, that still is a fair amount of words. You know, especially if you go for the option of, to what extent does this make you likely or unlikely, right? So there can be a fair amount of words. So is there a way to tackle rating-style questions that minimizes the number of words in the actual presentation to the respondent? Well, those of you who've taken questionnaire design training courses with me know I'm a big fan of semantic differential scales. With semantic differential scales, you're basically presenting a rating scale, but you're doing it purely with pairs. And so we might say, please rate this product using the pairs below. And each pair is typically presented on a seven-point scale. You do sometimes see semantic differential scales with different lengths, but most commonly it's seven points. And in the case that I have here, for those of you who are watching on YouTube, I'm showing please rate this product on several different items ranging from low-quality to high-quality, or low-priced to high-priced, or frivolous to sensible, typical to unique. So by presenting what I'm trying to capture, the attitude that I'm hoping my respondent is going to accurately self-report to me as pairs, I'm using a lot less words and everything is simply in adjective pairs. They're getting everything in a bipolar pair. And you can imagine that there's a lot of really great things you can do with this. For example, I've seen semantic differential scales used in customer SAT work where the pairs were things like terrible to delighted or frustrating versus satisfying, right? So there's almost an endless number of adjective pairs that you can come up with. Now, if you haven't used semantic differential scales before, I do recommend that you use pairs that have been tested. I always recommend the Handbook of Marketing Scales as a good resource to find various types of rating scales, including pairs for semantic differential scales. Otherwise, what I would say to you is, if you've got some cool ideas for pairs that you want to try, test them out before you roll them out in a real production survey or test them at least in your pretest to see whether or not they're working for you. But I like this approach as an option in cases where I'm really trying to reduce the number of words that my respondent is going to have to read and potentially be biased by. So semantic differential scales are really common. You do see them a lot in healthcare research. You see them sometimes in academic research. For some reason, we don't see them as commonly in commercial market research, but there's no reason for that other than the fact that I think a lot of professional researchers were sort of trained on Likert scales or Likert scales, right? So they're used to those five-point rating scales. But semantic differential scales are an option. So if you find yourself increasingly in situations where you're concerned about unintentionally biasing responses to rating scales, or you're really concerned about wordiness of how scales are presented, I definitely encourage you to check out semantic differential scales as an option. I hope that tip was helpful for you. And if you would like to share this video, share this YouTube with other folks, or if you're listening on iTunes, share this podcast with other folks, I really appreciate it. Do forward it, do like, and subscribe on either YouTubes or on iTunes. So thanks everybody for your help in keeping the conversation going, and I'll look forward to seeing you next week. Thanks everyone.