It’s not what you say; it’s how you say it. All of us have heard that expression at some point, but it’s especially important to consider when crafting questions for your market research survey. How you phrase your questions can influence a respondent’s answer, whether intentional or not. If the way you word a question impacts the respondent’s response in any way, you risk your data becoming inaccurate.
Well-written survey questions shouldn’t leave respondents confused about what answer they should give. If the question is written in a way that sways the respondent to one side or another, they cannot answer it 100% truthfully. For this reason, you should avoid using leading questions in surveys.
Table of contents:
Leading questions contain phrasing that can sway someone’s opinion in one direction or the other. Survey authors write these questions under the influence of personal biases and opinions.
For example, “Do you think X’s pesto sauce is the best?” is a leading survey question because the person asking the question expects the respondent to agree that their sauce is the best.
A more neutral way to ask this question is, “What do you think of X’s pesto sauce?” For many survey authors, it is difficult to avoid writing leading questions, but by using neutral language and adhering to the following steps, you can create an impactful survey that receives honest responses.
Leading questions are problematic in online surveys because they can introduce respondent bias, influencing how respondents interpret and answer a question. If responses are biased, the survey will fail to capture authentic feedback, leading to skewed results and inaccurate insights.
Further, leading questions can compromise respondent trust. If participants become aware of the bias in leading questions, they may view the survey as manipulative or untrustworthy.
Great survey questions avoid using persuasive language that can influence a respondent’s opinion. Sometimes, using neutral language can be tricky, especially when you are passionate about a specific product or service. Here are a few guidelines to bear in mind when authoring survey questions:
Neutrality and open-ended questions are the keys to avoiding biases in survey responses. It helps to involve a third party with market research experts to review your questions to ensure the tone remains neutral and unbiased. For example, IntelliSurvey’s services include a detailed questionnaire review to ensure your survey receives the best responses possible.
There are many different types of leading questions to watch out for when writing a survey. Since there are several kinds of leading survey questions, they can be harder to spot than you think. Learn more about the various types of leading questions to watch out for below.
Good survey questions should be written in a way that measures only one factor. Double-barreled questions are tricky because they give the respondent more than one element to consider. Since these questions measure two different things, they make it impossible for a respondent to answer accurately.
For example, “How satisfied or dissatisfied are you with your room and room service?” is a double-barreled question. In this example, the question is asking about two separate things: the room and the room service.
A respondent might not be satisfied with the room but find the room service attentive and helpful; therefore, they say they are more content than they are with the overall experience. See these additional example questions:
Yes or no questions that use absolutes like “always” or “never” should be avoided because they do not allow the respondents to give helpful feedback.
For example, instead of asking, “Do you always eat breakfast, lunch, and dinner? Yes or No?” ask, “What meals do you eat throughout the day? Check all that apply.”
Questions with absolute answers are too rigid to be effective in a survey and prevent the respondent from giving any feedback or suggestions to improve the product or service. Check out these additional examples:
Assumption-based leading survey questions wrongly operate on preconceived notions held by the survey’s author. This type of leading question is commonly seen on feedback surveys for products, where the survey author will ask, “How much did you enjoy using your product or service?”
In this example, the survey’s creator makes the assumption that the user or respondent enjoys the product to some extent. Avoid leading questions that make assumptions and give users the opportunity to say they dislike the product and provide reasons why. Some other examples include:
Direct implication leading questions are typically found in experienced-based surveys. This type of leading question usually uses an “if, then” statement. They lead with a hypothetical scenario to get respondents to consider their answer if something were to eventually occur.
For example, “If you found our service helpful, then would you use it again?” This type of question will yield a false response from the respondent because they would most likely say yes.
Instead, improve this question by including a specific function for them to consider, “if we implemented features to improve readability, would you consider using it again?”
Additional Direct Implication Questions:
Also referred to as tagged questions, coercive leading questions in surveys are aggressive questions that force respondents to respond in a way that is typically positive. This type of question is often found in customer satisfaction surveys.
For example, “You enjoyed your meal, didn’t you?” This question is worded in an intimidating fashion and will likely dissuade customers from returning to this restaurant. Coercive leading questions are fruitless and provide no meaningful way to measure customer satisfaction or receive helpful feedback.
Additional Coercive Leading Questions:
Scale-based leading survey questions are typically used in experience-based surveys to measure how pleased a customer is with a service or product. Scale-based questions use the Likert scale to give respondents a bigger range of answers to appropriately measure how they feel about a product or service. This type of survey question is effective when used correctly because it gives researchers deeper insights into how customers feel about the product.
The most common error survey authors make while creating scale-based questions is tipping it in favor of the company or researcher. For example, scales with more “satisfied” options than “dissatisfied” options.
Example: How satisfied are you with our product?
Since the scale is tipped on this survey question, respondents are more likely to be satisfied with their product on average, no matter what. This question will yield misleading results and statistics.
Avoid acronyms and abbreviations that may confuse the reader. Explain any definitions or concepts that an average member of the target audience may need to know to answer truthfully. Keeping language clear and concise can help prevent creating leading questions.
In the market research industry, different organizations often have different names for the same functional roles. What one organization may call a “Field Manager”, another might call a “Project Manager”, “Project Consultant”, or “Account Manager”. Surveys can be optimized by leveraging the title you believe will be clearest to the largest number of people and defining the role the first time it’s introduced.
Other Examples of Unclear or Drawn-Out Questions
While leading survey questions gently push a respondent toward feeling a certain way or selecting the desired response, loaded questions are trick questions no matter what. It doesn’t matter what the respondent says because they might be giving an affirming answer to something they do not agree with.
For example, a fast food chain asks, “Will you come back and enjoy another delicious meal?” Even if the respondent answers no, they still agree that the meal was delicious.
When creating a survey for a product or service you feel passionate about, it can be challenging to avoid letting your biases and opinions creep into the content. However, it is important to maintain a neutral position when crafting survey questions to ensure that you receive helpful answers and feedback.
Be sure to avoid using leading questions, coercive language, and other persuasive tactics when creating your surveys. It is often helpful to get a second opinion, both internally and externally. IntelliSurvey, for example, partners with our clients to review questionnaires and offer advice before programming a survey, helping ensure clean, reliable, and accurate results.
IntelliSurvey offers a variety of market research services and survey software to help companies conduct effective surveys and large multi-market studies. If you’re interested in working together, please contact us for more information.