Testing an online survey before fielding is essential to increase survey reliability and minimize the risk of inaccurate or inconsistent data collection. Through careful evaluation, you can find and fix issues before your survey goes live to ensure you get the most accurate results possible.
Here are several areas to test to ensure your survey is well-designed, functional, and capable of collecting reliable and valid data.
Questionnaire design and content are the heart and soul of your survey and should be tested with care. It’s surprisingly easy to unintentionally introduce response bias, inadvertently ask leading questions, make a mistake with screeners, or include other missteps. Thorough testing helps you catch and correct issues before going into field.
How to test: Carefully review your survey questions, response options, and question instructions for clarity, brevity, and relevance. When proofreading, be sure to include survey invitations, landing pages, and other related items. Ask a colleague or two to also review, as they might catch errors you did not see or interpret things differently than you intended.
Tip: IntelliBuilder allows you to update wording yourself without going back and forth with a programmer via Word doc. As you review the questionnaire, you can make direct edits and know they’ve been incorporated immediately.
If your survey contains skip logic or branching, test these features thoroughly to make sure respondents are directed to appropriate questions based on their responses. As you test each potential survey path, also review validation and error messaging. Note any issues such as broken links, missing images, or other technical issues.
How to test: Use a random data generator (also known as dummy data) to mimic various scenarios, including those that trigger different branches and skips in the survey. Manually walk through the survey and test all branches and logic paths. Make sure to test error handling, too, by intentionally providing incorrect or invalid responses during testing. Log any issues and send them to your programmer for resolution.
Tip: If you’re working with IntelliSurvey’s platform, you can save significant time by leaving comments for programmers in the survey as you test, making notes as you go without having to start over. You can also see when comments are resolved, so you know exactly where things stand at all times.
Ensure your survey is accessible to as many individuals as possible by testing with screen readers and other accessibility tools.
How to test: The Web Accessibility Initiative (WAI) provides links and resources to help you assess the accessibility of your content. If you’re using IS Pro’s default respondent theme, you’re set - it’s already ADA and WGCA compliant out of the box.
To maintain transparency and compliance with data protection regulations, be sure to test your consent and privacy links before fielding your survey.
How to test: Start by manually clicking the consent and privacy links within your survey invitations, landing page, and the survey itself. Verify they link to the correct destination, such as your privacy policy, terms of service, or other relevant documents. Next, test opt-out and withdrawal mechanisms to ensure they work as described.
If you’re using an online panel partner, it is essential to test the integration to establish a smooth and error-free data collection process between platforms.
How to test: Initiate a data transfer between your survey platform and the panel company’s platform. Ensure that respondent information and survey links are exchanged correctly. Using test accounts, complete the survey and confirm the response data is transmitted correctly between platforms.
Tip: IntelliSurvey has already set up connections with over 100 panel providers, so you can skip this step with confidence when working with us.
Test your survey on different devices and browsers to check for any display or functionality issues. Stay mindful of best practices to optimize mobile surveys, and don’t forget that brevity is better for smaller screens.
How to test: While testing with actual devices and browsers is best, mobile emulators and simulators can also allow you to view and interact with your survey as it would appear on various devices. Chrome DevTools and BrowserStack are two popular simulators.
If your survey involves distribution via email, social media, or other channels, test each channel independently to ensure links work and respondents can access and submit the survey easily. This is also the time to test that any QR codes, shortlinks, and tracking pixels are working as expected.
How to test: Create test contacts that will receive survey invitations or links during testing, then generate unique links/invites for each test contact. Verify link delivery and tracking mechanisms, such as UTM parameters appended to survey links. If you are using email invitations, verify they are arriving in the inbox and are not immediately routed to a SPAM folder.
Consider a soft launch to a small group of participants. Run a few basic data cuts or manually review respondent data to ensure data is collected as expected. This lets you catch any last-minute issues before opening your survey to the full pool of respondents.
How to test: Administer the entire survey to a small sample of respondents. Monitor the responses and look for patterns or issues. Make adjustments as needed.
Thorough testing is a critical part of the survey preparation process. In addition to harming the credibility of your survey, errors can also undermine its validity. Before you field your next survey, review this list of precautionary measures to avoid making costly errors. If you would like a trusted partner to support your efforts, please get in touch.