I have introduced laboratory experiments in the course of social research methods since 1959. Wording experiment is one of the most important among laboratory works in the course. I have imposed the students to make questions feasible to search (1) influence of stereotyped word, (2) difference between impersonalized and personalized questions, (3) inadequacy of double-barreled questions, (4) carry-over effect, (5) difference between multiple choice and free answer questions, (6) influence of difficult word to understand, and (8) yes-tendency. They have to interview around fifty other students, tabulate answers, and analyze them. I reanalyzed the data thirty nine working groups of students offered in the course in this paper.
Experimental wordings the students constructed within limitted hours, were not enough to detect the eight consequences mentioned above. Double-barreled question showed statistically significant effects most frequently, and impersonalized vs. personalized questions, multiple choice vs. free answer methods, ambiguous word, difficult word to understand, stereotyped word, and carry-over effect are less frequently in this order. The students failed in yes-tendency experiment most. Although such results partly depend upon the difficulty in constructing the artificial wordings, the order may suggest at the same time that there seldom happen yes-tendency, carry-over effect, and stereotyped word's influence, and you should pay attention most to double-barreled question, which can ofen occur.
Doble-barreled question has two types. The first type of the question A is like this: “do you agree to B 1 and B 2?” It should be divided into two separate questions: “do you agree to B 1?” and “do you agree to B 2?” Whether responses of the question A would be influenced more by B 1 or B 2, does not depend upon the order of B 1 and B 2 in the question, but upon the contents of B 1 and B 2. The second type of double-barreled question A' is like this: “do you agree to M because it is N?” It consists of a principal clause B 2 of “do you agree to M ?” and a subordinate clause B 1 of “is it N (in your opinion)?” Responses to the double-barreled question A' are influenced by B 2 more often than B 1, and there are adverse cases too. The content of B 1 and B 2 also affects the response of the question A', as in the case of the first type of double-barreled question A.
Difference between multiple choice and free answer questions results from two causes. The one is that alternatives in the former are often inexhaustive. Secondly, a particular answer may be incapable of being recalled in the free answer method without the alternative indicated. Consequently, the multiple choice question can be said superior to free answer method, if the former shows an exhaustive list of possible alternatives.
View full abstract