In the realm of survey research, ensuring data integrity is paramount. Door screening and speeding...
Have you ever wondered how survey fraud rates vary across sample provider types? While there is no one-size-fits-all answer, our recent experiment showed that the impact can be huge. In fact, some mainstream sample sources exhibit over double the amount of fraud than higher-quality alternatives.
Our Survey Panel Provider Quality Experiment
IntelliSurvey recently conducted an 8-minute survey to explore the quantitative research respondent experience in the United States.
For observation purposes, we allowed all respondents to complete the survey, in contrast to our usual live cleaning practices. A total of 2,000 surveys were collected from four major sources in the industry:
Panel A: a large, established mainstream panel company using managed services
Panel B1 (Direct): a large, tech-first panel company using managed services
Panel B2 (Aggregator): the same panel provider as B1 using their panels through an aggregator
Aggregator C: an aggregator sample, excluding B1
Once collected, we identified fraudulent respondents through a combination of IP and device finger-printing, automated in-survey behavior checks with our proprietary cleaning algorithm CheatSweep™, as well as manual consistency checks and open-end reviews.
On average, we found 31% fraud across all panel providers. However, not all sources were equal - that rate varied massively between vendors.
As shown in the chart below, aggregator quality was notably inferior compared to going direct to panels. When purchasing direct from a vendor, the data quality was significantly better, thus minimizing the time needed for data cleansing. Interestingly, the biggest brand (Panel A) did not offer the best quality. Panel B1 outperformed it.
It's worth noting that we did not explore any third-tier vendors in this study. However, in our rejected vendor pilots, we routinely encountered fraud rates exceeding a staggering 80%!
How IntelliSurvey Can Help
Do you have a clear understanding of the fraud levels associated with your panel sources? At IntelliSurvey, we have the expertise and tools to help you navigate this complex landscape. Make sure the data in your survey is accurate by getting in touch with us today.
Subscribe to our Monthly Newsletter
Analyzing brand funnel metrics is crucial for any business wanting to understand its target market...
Quantitative data is all around us. In the form of surveys and market research, this data type...