Speaker 1: Hey folks, Katherine here from Research Rockstar. Thanks for joining me here today. And today I would like to talk about online surveys. And in market research these days, of course, we have many methodologies available to us and whether we're doing quantitative or qualitative research, we have so many different methodologies, but surveys are definitely still a big part of what many of us in market research and customer insights do. So I would say personally for myself, over the last year, I have seen probably still a good 50% of the projects I'm seeing are based still on survey research. Now that's just my personal observation, but still, that's a lot of survey-based research going on out there in addition, of course, to focus groups and social listening studies and all sorts of other types of studies that we have these days with ethnography and self-ethnography and so on. So a lot of methodologies out there, but still a lot of survey research. But there can be a challenge with survey research. So have you ever done a survey research project and this is what you feel like? Well, it can be very, very frustrating. Survey research can be painful and my picture here of a gentleman in front of his computer, an obvious dismay is very typical. So where do things start to go wrong? So if somebody is doing online surveys and you're really unhappy as, you know, trying to get this great data in, what's usually the root cause? Well, there are a number of things that can go wrong when we're doing online surveys, but a lot of the pain that we experience is due to the fact that these days response rates for online surveys are very low. Some might say tragically low. And so when you are working on a big project and you're supposed to be delivering data and you see that the response rates are under 1%, that obviously makes you unhappy. There's the practical considerations of will you be able to get enough, will you be able to invite a large enough pool where 1% is going to get you the sample size you need, but it also of course raises the classic issue of non-response bias. If I'm doing a survey of our customers and less than 1% of them are taking our survey, how do I know that that group of people is really representative of my overall customer base? It really raises some very uncomfortable questions. So we know that one of the things we need to do is do everything we can to maximize response rates to our surveys. Now how do we do that? How can we help companies in our own work and working with different clients? How do we help get better response rates to online surveys? Well there are a lot of things that we do that are really well documented, right? We try to make sure that our surveys aren't too long. We make sure that we offer incentives whenever it's appropriate and we're allowed to. We offer incentives to people to take our surveys. And of course these days we also design our surveys to be mobile first. That is, we know that in the world of online surveys that most people take those surveys from a mobile device, not from a desktop, not even from a laptop computer. And so we know that if we want to have good response rates, we have to have a mobile first design. Well a lot of those issues are really well researched and well documented. But let me ask you a different question. When was the last time you really tried to improve your email invitation? Let's be honest, you probably have an email invitation that you've kind of standardized on over time. It's worked okay for you, it looks pretty professional, your colleagues like it. But when was the last time you really, really tried to improve it? And I think that there is opportunity here. So if you haven't tried to improve the actual email invitation itself in a while, let's take a look at some possibilities for improving your response rates by improving the email invitation itself. Now there have been a number of studies about different ways to potentially improve survey response rates by changing attributes of the email invitation itself. Now I'm going to give you some examples of things that you can try. And the reason why I'm saying you can try is because some of these variations, some of these modifications to survey invitation text and email format will work better with some demographic groups than with others. So if, for example, you do B2B research but you have a friend who does a lot of consumer research, some of these things may work for one of you but not the other. Or if I'm doing a lot of research with senior citizens and you're doing a lot of research with millennials, we may have different experiences as well. So that's why it's a good idea to experiment. If you do research with certain types of populations, it can be a good idea to experiment with those populations to see which of these specific ways of changing your email will actually get you a better response rate. So be cautious of general rules of thumb when it comes to improving response rates to online surveys by modifying the email because while I'm going to share with you some sources of experiments that were successful, that is that identified how specific email attributes can improve response rates or have in certain experiments improved response rates, there's also been some contradictory research. That is, one researcher has found that a certain change to an email actually improved response rates for them. Another researcher found it did not improve response rates. In my review of some of this research on research, what I find is they're also sometimes doing their research with different demographic groups and that really aligns with my own experience just in general with response rates, with what incentives work, et cetera. It really can vary a lot not only by the subject of what you're surveying about but the specific target population of interest. So these are some guidelines I'm going to give you based on research that's been done that has shown to have a positive impact on response rates, but again you need to experiment to find out if it will work for your specific population. So one of the cool ideas, place the survey URL near the top of the email. Don't feel like your email has to have like two paragraphs to warm them up before you actually give them the link. Put the link early on in the email and this was from an experiment that was done by the folks at ACT, the college testing company, in 2016. They had some success with that and I kind of like that because I do see a lot of survey invitations where the email, the link to the survey can be a little bit buried. So that could be a really easy thing to experiment with. How about this one? A couple of researchers from SurveyMonkey in 2016 found that embedding the first question in the invitation itself had a positive impact on response rates. Now that does require a little bit of work to embed the actual question itself and so that you capture it in the survey invitation, but it's certainly doable and again an experiment from the folks at SurveyMonkey found that it did have a positive impact on response rates. I'm not really sure why. I mean my speculation is that maybe when some, maybe when people saw that question and the question was reasonably interesting perhaps or it was short that they thought, oh this will be okay, it's not going to be too painful so they clicked on it. That's just my speculation. But since there has been some successful research on that, I do think it may be worth experimenting. How about this one? Try a more authoritative subject line. So again this was research from the folks at ACT. They by the way have a lot of great research on survey design on their website. They do a lot of research on research because they do a lot of research with college students and so in that case that might be something like instead of having a subject line that's like please we'd love your feedback on blah blah blah, it's the executive vice president of product development would love your opinion. So it's putting that somebody important in the subject line or somebody of authority in the subject line or instead of it being a sort of a soft we'd like your opinion, maybe it's something like this research is being done to make a million dollar investment decision. So something with authority, something with some weight to it may have a better impact than a generic please take our survey kind of subject line. So again do experiment from that. Send from a known sender. So if you are sending out survey invitations and nobody knows who you are, that could have a negative impact on your response rates. And so having the send person be somebody who's known may have a positive effect. There was some interesting research on this from some researchers at Arkansas State University and Morgan State University on this in 2017. So definitely worth trying out. So if you can say, like let's imagine you work for a company that has a really well known CEO. Can you use the CEO's email address and have it come from them? Is that something that would help with that response rate? Or if you're doing research for an organization that has a really, you know, where you're doing a survey of customers and a lot of customers would have known or heard of the VP of sales, can you use her name? But having it come from a known name is something that can really help people to respond. Now something I've often done on the B2B side when I've done B2B research is I would sometimes do a survey invitation where I'll first have a warm up email go out from the CEO or the VP of sales or marketing. So first have just a letter from an executive at the company go out saying, hey, you're going to be hearing from my colleague who's doing a study for us. I'd personally be very grateful if you participated, blah, blah, blah. Then I follow up with my survey invitation the next day and people are kind of warmed up. I found that actually worked really well for me. Another experiment that you can try is to vary whether or not your text uses the word survey in it. Some people in their invitation emails actually have avoided the use of the word survey because they feel like it has potentially a negative halo effect and so they might say we'd like your feedback or we want your opinion, but they sometimes avoid the phrase survey. But actually, again from our friends over at ACT, they actually found that putting the word survey in the text helped with response rates. So again, that would be an interesting experiment. Take that experiment where you've got one version of your invitation that includes the word survey and one that doesn't. One last experiment is about actually not the text for the invitation itself, but for the reminder email. When you send the reminder email, consider mentioning the actual incentive in the reminder email subject line. Research by some researchers at the University of British Columbia in 2014 found that reminder emails were significantly more successful in boosting response rates when they actually included the incentive amount. So that's something worth trying. So again, try the URL near the top of your email, embed the first question in the invitation, try a more authoritative subject line, not a wishy-washy one, send from a known sender, do use the word survey, and consider mentioning incentive in the reminder email subject lines. Now I know those are a lot of different attributes, you're not going to be able to do all of those experiments at once, but maybe try one or two of these things. The next time you're doing an online survey, maybe split your population into three buckets and test the three different treatments and see, did one bucket of people get a higher response rate than the others? It's certainly worth an experiment. And by the way, when you're thinking about experiments, think about some of the great lessons we can also sort of borrow from our colleagues from the email marketing world. There's a lot more experiment data from email marketers than from market researchers, and so a couple of the things that I think are really interesting and I personally would love to experiment with myself, one is the idea of embedding images into the survey invitation email. Research about email marketing effectiveness has shown that images are really important and are really important in social marketing as well. So if you add an image, for example, to a tweet, the tweet is going to get a lot more clicks. Facebook posts that have images get 2.3 times more likes than those that don't, and specifically images with faces. So connecting maybe a little bit. So again, we know that from the world of social media marketing that tweets and Facebook posts with images seem to get a lot more attraction. Is that something that we can take advantage of as well? How about subject line length? Email marketing consultants generally will tell you that six to ten words is ideal. You need enough words to kind of hook them in and be precise and set some expectations, but too many words and people actually get turned off. So again, maybe testing whether or not an image in your email is going to help or whether a variation on subject line length would help. It could be very interesting as well, and there's tons of research from organizations like HubSpot and MailChimp and Constant Contact. All of these companies that are really in email marketing, they actually publish a lot of research on what helps to get email read, which is what our online survey invitation is, right? It's an email and we want people to read it. I hope the conversation was helpful today. I hope everybody tries some experiments, tries to improve their survey invitation text. If you do some experiments and you're willing to share the results, please do so. Share your results with the rest of the market research and insights community on LinkedIn and some of the discussion groups. Share it here, share it with me, and I'll help publicize the results, but the more we can share about learnings, the better it's going to be for all of us and hopefully help to improve response rates by improving our invitation text. Thanks everybody for being here today. If this was useful, please do like this video and subscribe. And for those of you who do take classes at Research Rockstar, I just want to mention that we are having this class coming up in a few weeks with Dan Reynolds, who's one of our most popular instructors here at Research Rockstar. Some of you may have taken his Excel for Survey Data Analysis class, which is really, really very popular. And he's going to be teaching our new Writing for Impact Boot Camp for market researchers. Now we've been talking a little bit about writing today. Writing skills are important. And the crazy thing is that these days, for us, those of us in market research and insights, we're always being pushed to write shorter and shorter reports. You know, people are busy, they want us to write very short form documents, but that actually makes writing skills even more important. If I don't have that much of somebody's attention, I have to make sure that what they read from me is incredibly well written and impactful. So writing skills are super important these days. And in this class, Professor Dan takes you through a great way of refreshing on some writing basics and then tackling specific writing challenges that are common in market research writing. So in this class, he uses examples from reports, from top lines, from client emails, all sorts of specific things that market researchers have to write to give you practical lessons. So do check out that class. It starts April 19th. And again, please do like and subscribe and we'll chat with you next week. Thanks everyone.