7 B2B Survey Design Pitfalls and How to Avoid Them (Full Transcript)

[00:00:06] Speaker 1: Hi friends and welcome. Let's start with a question. Which do you think is harder to design, business-to-business surveys or business-to-consumer surveys? Many people instinctively say they're equally difficult, and at a high level that's true. But once we get into real-world research projects, we quickly discover that business surveys present some unique challenges. In this conversation for Research Rockstars, we'll walk through questionnaire design considerations that protect both data quality and respondent experience when conducting surveys of business professionals. Of course, the basic principles of good survey design apply to all surveys. No matter what our population of interest is for a given project, long or confusing questionnaires increase dropouts and lower data quality. But in consumer research, we typically have such a large pool of people available that if we collect some bad data, we can identify and replace it. But when surveying businesses, our size of universe is much smaller, so we have to be extra careful to avoid losing qualified and willing survey takers. Challenge number one. Sample cost. In business research, qualified respondents are often expensive to reach. Unless we have access to a client's customer list, which is not always possible, costs add up quickly. Online panel completes for business participants can easily exceed $30 per complete, and sometimes much, much more. That means every respondent is extremely valuable. Here's an example. Imagine we are studying commercial painting businesses and how they choose interior paint brands. We do not just need painters. We need the people who control paint brand selection decisions. Those people are harder to find and costlier to recruit than consumers who buy paint for home projects. Also they are busy professionals, often at job sites or driving between job sites, which makes them hard to reach. Because of all this, we need to treat these qualified respondents like gold. Keep questions short and clear. Challenge number two. Anonymity concerns. Many business participants who decline survey invitations do so for the same reason. They are worried their responses will not be anonymous. Worse, they sometimes worry their responses could be traced back to their employer, and that can feel like job risk. To protect response rates, we need to address privacy and anonymity up front. Emphasize that responses are confidential and that no personally identifiable data will be linked to survey answers. Another tactic. Consider offering multiple participation modes, such as online, phone, or even paper, so respondents can choose the format that feels most comfortable and private to them. Yes, multimode research takes extra effort, but if it means getting better participation from qualified business professionals, it can be very worthwhile. Challenge number three. Survey length. Because business-to-business panel costs are high, some clients feel compelled to get their money's worth by asking more questions. But that approach backfires. Longer surveys drive up costs further and increase the risk of satisficing and dropouts. Part of our job is to remind clients that data quality depends on keeping surveys reasonable in length. When a client insists on extra nice-to-know questions, gently ask, Would you personally complete this questionnaire? That usually helps reset priorities. Challenge number four. Personal versus company opinion. When doing surveys with business people, be precise about which perspective you want—personal views or company policy. For example, imagine asking our painting contractors, To what extent is fast drying time a consideration when buying interior paint? The respondent might personally care about drying time a lot, but know that management prioritizes consistency of application. Both answers could be valid, but without knowing which point of view is needed, you will get muddy data. To avoid this, we can use wording that clearly signals the level of response needed. For example, saying, You personally for individual perspectives and saying, Your company for organizational policy or behavior. Challenge number five. Buyers versus users. In business research, product buyers and product users are often different people. They have different goals and knowledge levels. Because of this, we have to be clear about which role our study needs. Do we need the buyer with purchasing authority or the person who actually uses the product? This distinction exists in consumer research, too. A parent might buy a gaming console and be a good fit for a survey about price sensitivity and purchase experience. But the child uses the gaming console and would be a better fit for a survey about feature requirements. In any case, we want to be clear about the role we need to reach, so we can align our sample sources and qualification criteria accordingly. Challenge number six. Company versus division focus. Many business topics can apply to an entire company or only to one division. If that focus is not stated clearly, you risk collecting data from the wrong unit of analysis. For example, a sales manager answering questions about your company might actually be describing only their experience within a division or geographic region. To avoid this risk, we have to word questions carefully. We might say, For these questions, please consider the entire company including all locations. Or conversely, For these questions, please base your responses on your specific department or division. And remember, we can always offer not applicable or do not know options to help respondents avoid guessing. Challenge number seven. Long screeners. Business surveys typically need longer screeners than consumer surveys. That's because we screen for both individual and company attributes. Individual attributes might include job title, job role, tenure, or decision-making authority. Company attributes often include industry, size, number of locations, or revenue. All of that takes time in our survey, and so it reduces the time left for core questions. Now, you might be wondering, why would we ask for both job title and job role? Titles alone often are not enough. For example, a chief executive officer at a small company may purchase office supplies themselves, but a chief executive officer at a large firm does not. Same title, totally different things they can speak to. These differences are magnified in multinational studies, where job titles often have very different meanings across countries. So yes, screeners for business surveys take longer, and that is often necessary. What that means for us as researchers is that in a questionnaire, we may not have as much time for our core questions as we would like. Consider the example of a 10-minute business survey, where we might have time for 25 questions overall. We may reasonably need 8 screening questions. That leaves us just 17 for our main topic. So when selecting screening criteria and writing those questions, we have to make very careful choices to enforce what we need without wasting any questions. That wraps up our discussion on business-to-business survey design challenges. If you have any questions, please post them in the comments section or email us at info at researchrockstar.com. And thanks for joining us for this episode of Conversations for Research Rockstars. We hope this topic was useful to your work, and if it was, please like and subscribe. That helps us share this content with more of your fellow research rockstars.