Careless Responding in Survey Research: An Examination of Individual, Situational, and Cultural Characteristics

Abstract

Careless responding is the act of responding to measures with low or little motivation in the form of not properly complying with survey instructions, interpreting survey items, or providing accurate responses (Huang et al., 2012; Meade & Craig, 2012). Careless responding is a concern for all researchers that implement any form of self-report survey methodology within their research as it can have varying effects on data quality and overall data interpretation (Credé, 2010; DeSimone & Harms, 2018; Huang et al., 2015; McGonagle et al., 2016). A comparatively large proporiton of research has examined response bias, whereas a limited, but growing, literature base still lacks a completed understanding of careless responding. This study proposes a unified framework suggesting careless responding is a funciton of respondent characteristics (e.g., age, educaiton, individual values), situaitonal/survey characteristics (e.g., interview distractions, respondent interest), and cultural characteristics (e.g., cultural values, cultural norms). To examine the conceptual framework, mutlilevel analyses using the European Social Survey (N = 30,918) examined careless responding across 23 countries. This dataset allowed for the examinaiton of participants from a naitonally representative cross naitonal study implementing a standardized face-to-face interview based survey. This study found that careless responding was higher among participants who had a stronger value of Openness to Change, Self-Enhancement, and Conservation values. Whereas careless responding was lower among participants who had stronger value of Self-Transcendence. Careless responding was also higher for participants who were older and had less years of education. Additionally, participants with higher perceived interest in the survey had lower leveles of careless responding. When distractions were noted during the survey, in the form of having outside members present or interfering with the survey process, careless responding rates were higher. The relationship between distraciton and careless responding disappeared when accounting for other individual and situational/survey characteristics. Additionally, careless responding was lower in countries with tighter cultural norms, but this association disappeared after considering respondent and situational/survey characteristics. These findings provide initial support for the proposed conceptual framework that respondent, situational/survey, and cultural characteristics play a significant role in careless responding. This suggests not only an opportunity to further examine these different multilevel characteristics, but that researchers implementing self-report survey methodologies should consider how individual, situational, survey and cultural characteristics may influence careless responding within their research.