In this study, we verify the difficulties of conducting web surveys, particularly ones that include narrative texts, and investigate some remedies. We identify the risk that the reading comprehension for the narrative texts in web surveys may be insufficient compared to that of face-to-face surveys, which may be an effect of effort minimizing, or satisficing, observed in prior studies. To reduce such effort-minimizing effects, we suggest some remedies; namely, (1) improved instructions, 2) screening by Instructional Question Check (IQC) whereby an instruction is inserted within the narrative text that requires a counterintuitive answer to the following question, 3) two-step survey designs, and 4) increasing the number of comprehension-checking questions. The results of this study indicate these remedies have a certain degree of effectiveness when conducting surveys with participants who are likely to appropriately comprehend the narrative texts.

 In this study, we examine whether an “Instructional Question Check (IQC)”, an approach to incorporating instructions that participants must respond to within the narrative texts presented to them, is effective as a measure of how attentively the participants of web surveys read compared to the results observed by Tsunemi et al. (in press). We also examine the variables that influence the instructions themselves, such as the appropriateness of reading times for the narrative texts, the types of devices employed, reading experiences, pleasure from reading the narrative texts, Japanese-language proficiencies, reading habits, and participant characteristics. The results indicate that an IQC is an effective measure of attentive reading and that reading times, reading experiences, and device type all had strong influences on the IQC pass ratio. These findings suggest that, for survey data that does not include IQC, it is vital to check for straightlining responses and to improve data quality based on appropriate reading times and reading experiences.