Effect of question formats on item endorsement rates in Web surveys: Comparing check-all and forced-choice question formats

Abstract

The purpose of this study is to test whether multiple-answer formats (MA) and forced-choice formats (FC) produce similar results in Web surveys. Data were based on a Web survey of 1,559 Japanese adults in the Tokyo metropolitan area in March 2010. The results revealed the following: (1) Respondents endorse fewer options and take less time to answer in MA than in FC. (2) For MA respondents, options are more likely to be endorsed when they appear in the first half of a list than in the second half. These findings suggest that MA may encourage weak satisficing response strategies. In addition, these tendencies can be seen not only in attitudinal questions (judgment-type questions), but also behavioral questions (recall-type questions). However, the differences between FC and MA are greater in attitudinal questions than in behavioral questions.