Crafting effective Likert scale questions is essential for gathering reliable data in surveys. However, even experienced researchers can make mistakes. In this post, we’ll explore five common errors that can skew your data and reduce your survey’s effectiveness.
Respondents need straightforward labels on Likert scales to properly answer the question. If there is any confusion, their answers won’t reflect their true feelings. One way to add more clarity is to use descriptive words instead of numbers. For example, when asking an agree/disagree question, instead of a scale of 1-7, your answer choices could range from Strongly Agree to Strongly Disagree.
Be sure to avoid ambiguous scale labels. When asking respondents, “How satisfied were you with the speed of service?” using terms like “very much” or “not very much” is less clear than “very satisfied” or “not very satisfied.”
When using Likert scales, they need to be consistent throughout your survey, such as always showing them as Strongly Agree to Strongly Disagree and not flipping one question to show the opposite. Consistency considerations include:
Inconsistent use can confuse respondents as they move from question to question. They also make it more complicated to compare responses directly during survey analysis.
Unclear questions can confuse respondents, leading to unreliable answers and undermining the usefulness of the results. Ambiguous questions include queries that:
Have more than one meaning: People can interpret the same word or phrase differently. “How expensive do you feel this product is compared to competitors?” The word expensive is very subjective and can be thought of differently by each respondent.
Try to measure more than one element together: Also known as double-barreled questions, these questions include more than one factor for the respondent to consider. “How satisfied are you with the price and quality of this product?’ Someone might be very satisfied with the price but unsatisfied with the quality. In this case, the question should be split into two.
Use jargon or language that is difficult for the average person to interpret: “How would you assess the qualitative attributes of the materials employed in the fabrication of our products?” is much more difficult to understand than “How do you rate the quality of the materials used in our products?”
Too many or too few scale points can impact your data collection, causing:
Lack of clarity: Too many scale points make it hard to distinguish between closely related options, while too few scale points may not capture the full range of opinions.
Central tendency bias: With a large number of options, respondents may tend to select the middle point, especially if they are unsure or indifferent.
While there is no “right” number of scale points for every question, studies suggest the optimal number is 5-7 to strike a balance between capturing accurate, meaningful data without overwhelming or oversimplifying respondent choices (NPS excluded).
Your wording can influence a respondent’s answers, either subtly or overtly leading them to a specific response. The question “How strongly do you agree that our innovative product greatly improves your productivity?” uses leading language (“innovative”) and also presumes a positive impact (“greatly improves your productivity”). To maintain neutrality, the question could be rephrased as: “To what extent do you agree or disagree that our product impacts your productivity?”
To avoid biased language in your Likert scale questions, use neutral, specific wording without leading phrasing, and ensure a balanced range of response options.
In summary, avoiding these common mistakes will ultimately help you collect more accurate and meaningful data from your survey. If you’d like help refining your Likert scales and overall questionnaire design, our dedicated research team is available to help.
With over two decades of experience in online survey programming, fielding, and analysis, IntelliSurvey is well-positioned to help you improve the quality of your study. Please contact us today to learn how we can partner to develop stronger outcomes for your research objectives.