Behavioral Economics Interventions to Reduce Insufficient Effort Responses in Open-Ended Questions of Online Surveys: A Randomized Controlled Trial

Abstract

The increasing use of open-ended questions in marketing research, particularly for collecting opinions on advertising videos, has raised concerns about insufficient effort responses (IERs), such as meaningless character strings. This study examines the effectiveness of three behavioral economics interventions (nudges) in mitigating IERs through a randomized controlled trial (n=10,324) conducted in an online survey. The interventions include (1) commitment, (2) a loss-framed message, and (3) a gain-framed message. All three interventions reduced both the proportion of respondents providing IERs and the total number of IERs. The loss-framed and gain-framed messages further decreased the selection of intermediate responses, which are commonly associated with low-effort answering. Importantly, no adverse effects, such as increased survey dropout rates or heightened response burden, were observed. On the contrary, the gain-framed message led to a reduction in both dropout rates and perceived response burden. These findings suggest that behavioral economics interventions can enhance data quality in online surveys by reducing inefficiencies caused by IERs in responses to open-ended questions.