The Modern Slump in Data Quality | IntelliSurvey

Surveys are incredibly valuable for businesses, helping them drive growth through understanding.

Decision makers expect these insights to come from honest, authentic respondents who represent a target population. However, this assumption does not always hold. Finding the right survey respondents has become an immense undertaking.

In the modern survey data landscape, studies are contaminated with fake data from survey farms, bots, and other types of malicious actors. Additionally, researchers don’t always have access to the right audiences. Worse still, some surveys are poorly designed, failing to engage respondents, or are fraught with response bias.

Even when research organizations follow what they believe to be the right methodologies, bad data can find its way into survey results.

For over 20 years, IntelliSurvey has helped brands launch tens of thousands of surveys resulting in around 20 million survey completions. Our experience minimizing the prevalence of bad data in our clients’ research led us to create an in-depth e-book, Overcoming the Biggest Threats to Market Research: Bad Data & Bad Actors, detailing the evolution of survey data, survey panels and bad actors, and various approaches to data cleaning. At the end of the e-book, you’ll find actionable recommendations you can implement to minimize the effects of bad data on your research and decision-making. Over the next four weeks, we’ll break down four key areas to understanding and protecting against bad data, including:

• The Importance of Data Accuracy in Business Intelligence
• Fighting Survey Farms: The Battle for Accurate Business Intelligence
• Low Incidence Can Be a Big Source of Bad Data
• How Experts Are Optimizing Survey Data Optimization

Why Has Data Quality from Survey Results Gone Downhill?

As mentioned above, survey contamination comes from several sources. As the demand for survey data has grown, so too have the incentives that come with a successful survey ‘complete’. This has attracted groves of new sample companies and, unfortunately, fraudsters. While some of the increase in supply is good, researchers now face new data quality challenges.

Bad Actors

Inauthentic and dishonest respondents often lie about their qualifications to participate in a survey.

This can manifest in several ways:

• Individuals looking for the quickest path to the reward without providing thoughtful, accurate answers
• Coordinated survey farms, which are fraudulent teams working on a larger scale
• Fraudulent bots, which are designed to finish surveys while appearing as natural human participants

Access to the Wrong Audience

When researchers don’t reach their specific groups or demographics, insights won’t reflect the true nature of the target population.

There are a few ways that researchers can end up accessing the wrong audience:

• Convenience sampling, like tapping a brand’s social media followers to participate in a survey
• Self-selection, which leads to survey respondents who have a strong bias in one direction or the other – as well as a higher probability of malicious actors getting into the mix
• Insufficient sample size, which can be heavily skewed by fraudulent actors making up a larger portion of the answers
• Using just one sample supplier, which reduces the survey footprint

To prevent the situations described above, it’s critical for researchers to vet and choose sample suppliers who not only have access to the right audience, but also have demonstrable sample quality.

Failing to Design Against Response Biases

Survey data can be contaminated when researchers fail to design their survey questions in a manner that lessens the impact of response biases, like:

• Non-response biases, where some groups of participants fail to complete surveys at higher rates than others
• Observation biases, where respondents skew answers to align with the conclusion they perceive the researchers want
• Social desirability biases, where answers reflect popular opinion more than the respondent’s true perspective
• Acquiescence biases, where respondents are more agreeable in answers than they actually are in the real world

Minimizing Bad Data in Survey Results

Intentional or not, bad data always finds new ways to contaminate survey results. Fortunately, innovations in survey design and data cleaning can reduce the impact of bad data on insights.

To learn about the latest in modern data cleaning approaches and for tips you can implement today to improve your data quality, check out our e-book Overcoming the Biggest Threats to Market Research: Bad Data & Bad Actors.