Net promoter scores (NPS) are a popular method used to measure customer experience with a company’s...
It’s not what you say; it’s how you say it. All of us have heard that expression at some point, but it’s especially important to consider when crafting questions for your survey. How you phrase your questions can influence a respondent’s answer, whether intentional or not. If the way you word a question impacts the respondent’s response in any way, you risk your data becoming inaccurate.
Well-written survey questions shouldn’t leave respondents confused about what answer they should give. If the question is written in a way that sways the respondent to one side or another, they cannot answer it 100% truthfully. For this reason, you should avoid using leading questions in surveys.
Read on to learn more about what these questions are, how to avoid them, and some examples of leading survey questions to look out for.
What Are Leading Questions?
Leading questions are questions that contain phrasing to sway someone’s opinion in one direction or the other. Survey authors write these questions under the influence of personal biases and opinions.
For example, “do you think X’s pesto sauce is the best?” is a leading survey question because the person asking the question expects the respondent to agree that their sauce is the best.
A more neutral way to ask this question is, “what do you think of X’s pesto sauce?” For many survey authors, it is difficult to avoid writing leading questions, but by using neutral language and adhering to the following steps, you can create an impactful survey that receives honest responses.
How Do You Avoid Leading Questions
Great survey questions avoid using persuasive language that can influence a respondent’s opinion. Sometimes using neutral language can be tricky, especially when you are passionate about a specific product or service. Here are a few guidelines to bear in mind when authoring survey questions:
- Keep your survey questions clear and straightforward.
- Use neutral language; don’t lead the respondent to a specific answer, conclusion, or opinion.
- Provide all possible answers to a question if using a multiple-choice format.
- If applicable, implement an “other” option and allow the respondent to enter their own response.
- Ask a third party to review your survey before sending it out; someone from outside the organization or department can provide you with a neutral perspective.
Neutrality and open-ended questions are the keys to avoiding biases in survey responses. It helps to involve a third party with market research experts to review your questions to ensure the tone remains neutral and unbiased. For example, IntelliSurvey’s services include a detailed questionnaire review to ensure your survey receives the best responses possible.
What Types of Leading Questions Are There?
There are many different types of leading questions to watch out for when writing a survey. Since there are several kinds of leading survey questions, they can be harder to spot than you think. Learn more about the various types of leading questions to watch out for below.
Good survey questions should be written in a way that measures only one factor. Double-barreled questions are tricky because they give the respondent more than one element to consider. Since these questions measure two different things, they make it impossible for a respondent to answer accurately.
For example, “how satisfied or dissatisfied are you with your room and room service?” is a double-barreled question. In this example, the question is asking about two separate things: the room and the room service.
A respondent might not be satisfied with the room but finds the room service attentive and helpful; therefore, they say they are more content than they are with the overall experience.
Questions With Absolutes
Yes or no questions that use absolutes like “always” or “never” should be avoided because they do not allow the respondents to give helpful feedback.
For example, instead of asking, “do you always eat breakfast, lunch, and dinner? Yes or No?” ask, “what meals do you eat throughout the day? Check all that apply.”
Questions with absolute answers are too rigid to be effective in a survey, and prevent the respondent from giving any feedback or suggestions to improve the product or service.
Assumption-based Leading Questions
Assumption-based leading survey questions wrongly operate on preconceived notions held by the survey’s author. This type of leading question is commonly seen on feedback surveys for products, where the survey author will ask, “how much did you enjoy using your product or service?”
In this example, the survey’s creator makes the assumption that the user or respondent enjoys the product to some extent. Avoid leading questions that make assumptions and give users the opportunity to say they dislike the product and provide reasons why.
Direct implication leading questions are typically found in experienced-based surveys. This type of leading question usually uses an “if, then” statement. They lead with a hypothetical scenario to get respondents to consider their answer if something were to eventually occur.
For example, “if you found our service helpful, then would you use it again?” This type of question will yield a false response from the respondent because they would most likely say yes.
Instead, improve this question by including a specific function for them to consider, “if we implemented features to improve readability, would you consider using it again?”
Coercive Leading Questions
Also referred to as tagged questions, coercive leading questions in surveys are aggressive questions that force respondents to respond in a way that is typically positive. This type of question is often found in customer satisfaction surveys.
For example, “you enjoyed your meal, didn’t you?” This question is worded in an intimidating fashion and will likely dissuade customers from returning to this restaurant. Coercive leading questions are fruitless and provide no meaningful way to measure customer satisfaction or receive helpful feedback.
Scale-based leading survey questions are typically used in experience-based surveys to measure how pleased a customer is with a service or product. Scale-based questions use the Likert scale to give respondents a bigger range of answers to appropriately measure how they feel about a product or service. This type of survey question is effective when used correctly because it gives researchers deeper insights into how customers feel about the product.
The most common error survey authors make while creating scale-based questions is tipping it in favor of the company or researcher. For example, scales with more “satisfied” options than “dissatisfied” options.
Example: How satisfied are you with our product?
- Extremely satisfied
- Somewhat satisfied
- Somewhat dissatisfied
Since the scale is tipped on this survey question, it is more likely that respondents will be satisfied with their product on average, no matter what. This question will yield misleading results and statistics.
How Are Leading Questions Different From Loaded Questions?
While leading survey questions gently push a respondent toward feeling a certain way or selecting the desired response, loaded questions are trick questions no matter what. It doesn’t matter what the respondent says because they might be giving an affirming answer to something they do not agree with.
For example, a fast food chain asks, “will you come back and enjoy another delicious meal?” Even if the respondent answers no, they still agree that the meal was delicious.
How IntelliSurvey Can Help
When creating a survey for a product or service you feel passionate about, it can be challenging to avoid letting your biases and opinions creep into the content. However, it is important to maintain a neutral position when crafting survey questions to ensure that you receive helpful answers and feedback.
Be sure to avoid using coercive language and other persuasive tactics when creating your surveys and get a second opinion from a different department or an outside organization, like IntelliSurvey, that specializes in creating effective surveys and studies.
IntelliSurvey has a variety of products and services to help companies conduct effective surveys and large multi-market studies. If you’re interested in working together, please contact us for more information.