The impact of direct question format on the detection of sensitivity bias using list experiments

Abstract

Sensitivity bias (SB) is a common issue when survey respondents falsify their attitudes and behaviors in response to sensitive questions. The list experiment is an indirect measurement method that aims to circumvent SB by asking respondents to report the number of applicable items among multiple items, rather than by asking about their agreement on a sensitive item directly (DQ). Moreover, SB can be evaluated by defining it as the difference between the estimates obtained through the list experiment and DQ. Despite the methodology of list experiments being well established, little attention has been paid to the different formats of DQ. This study addresses this gap by conducting three experiments. The findings show that the selection rate of sensitive items is significantly lower when DQ is measured with the multiple-answer format (MA) compared to the single-answer format (SA). However, there is no significant difference between the selection rate with single and multiple SA. These results highlight the importance of the DQ format in evaluating SB using list experiments.