Reproducibility and Validity of a Self-administered Food Frequency Questionnaire Used in the JACC Study

BACKGROUND: A self-administered questionnaire on dietary habits used in the JACC Study contained a 40-item food frequency questionnaire (FFQ). Although more than 110 thousand subjects enrolled in JACC Study and responded to the FFQ, no validation study has been conducted to date. METHODS: Eighty-five volunteers among the cohort members completed 2 FFQs (FFQs 1&2) and 12-day weighed dietary records (WDR). The interval between the two FFQs was one year. During the one year, the subjects carried out a 3-consecutive-day WDR in each season. We tested the reproducibility by using two FFQs. Also, we tested the validity of the FFQ by using the 12-day WDR as a gold standard. RESULTS: The intake frequencies of the 2 FFQs often agreed, showing the Spearman correlation coefficients ranging from 0.42 (edible wild plants) to 0.86 (coffee). The Spearman correlation coefficients of the energy and nutrient intakes from FFQ2, and that of the 12-day WDR were 0.20(energy) to 0.46 (animal protein, potassium). After adjusting the energy intake, the correlation coefficients showed 0.21(fish fat) to 0.51(animal fat). When classifying the FFQ2 and WDR by quartiles and examining the degree of agreement between the two methods, we obtained its median 30%. CONCLUSIONS: The FFQ is suitable to deal with a large group of subjects. However, since the energy and the amount of nutrient intake from this FFQ can not show the overall dietary intake situation, the subjects’ dietary intake should be assessed by categories.

"The Japan Collaborative Cohort Study (JACC Study) for Evaluation of Cancer Risk Sponsored by the Ministry of Education, Science, Sports and Culture of Japan (Monbusho)" is a large-scale population-based cohort study which launched in 45 municipalities all over Japan. 1 Epidemiologic data were collected at baseline (1988-1990) through a self-administered questionnaire with the written informed consent of the respondents, including such demographic information as sex, date of birth, marital status, number of children and medical history.Questions on their dietary habits were also included, such as the bowls of steamed rice they ate daily; the intake frequency of miso (traditional Japanese soybean paste) soup; the amount of its daily consumption in bowls at baseline.There were also questions about the intake frequency concerning 33 items with the use of 5 response categories, and those concerning alcoholic beverages and soft drinks.
In many large-scale population based on nutritional epidemiologic studies for analyzing the relation of nutrient intakes and health outcomes, a self-administered semi-quantitative food frequency questionnaire (SQFFQ) was used. 2 SQFFQ is composed of a food list and a frequency response section in which subjects are asked to report how often they consumed foods daily.This is the method by which the daily intake of nutrients was calculated by multiplying the intake frequency by the nutrients contained in portion sizes.However, in Japan little attention has been paid so far to the validity of the method of computing the amount of nutrient intake this way 3 .Moreover, although the intake frequency of specific food items or food groups is used to clarify the relation between the frequency and the incidence of diseases, a validation study has not been conducted, based on the actual intake frequency counted from the multiple-day dietary records.
We developed the method of evaluating the frequency and amount of consumed steamed rice, miso soup and beverages, and 33 food items.We tested the reproducibility and validity of frequency response and the method of estimating the amount of nutrients.
With respect to the intake amount of consumed steamed rice, replies were made in terms of the number of medium-sized bowls consumed.Hence, one such bowl was estimated to contain 140 g.The amount was thus obtained by multiplying 140 g by the number of bowls.The nutrient value of steamed rice is based on the Standard Tables of Food Composition in Japan, 4th revised edition. 4s for other foods, there were no questions about portion sizes in the questionnaires.However, we estimated them from the WDR results.That is, we computed the daily amount of food eaten by each person, based on the data from WDR, and determined the median as portion sizes (Table 1).As for food items without any specification, we decided on their nutrient composition as follows: there was no mention of the kinds of beef (e.g., sirloin, fillet) in the questionnaire, although various parts of beef were referred to in WDR.We selected them and computed those weighted mean ingredient values, based on the Standard Tables of Food Composition in Japan, 4th revised edition. 4The nutrient composition of foods other than beef was calculated in the same way.
As for butter and margarine, we calculated the median portion size by the same method as mentioned above.In Japan, few people use butter or margarine for cooking.Instead, they spread it on toast.We calculated the median portion size of bread, which turned out be 60 g.We added its nutrient value to the nutritive composition of butter and margarine.
When any particular item was used as a recipe in the questionnaire (such as deep fried foods or tempura, fried vegetables, pickles, preserved foods using soy sauce, boiled beans, etc.), we computed the median daily amount of cooking based on WDR by selecting the code in the names.The nutrient composition value was computed, based on the weighted mean value of all foods used for such cooking.
We calculated the intake amount of coffee, black tea, Japanese green tea and Chinese tea by multiplying 100 g per cup by the number of cups.For their nutrient values, we used the coffee infusion, black tea infusion, coarse-grade Japanese tea infusion and oolong tea infusion as listed in the Standard Tables of Food Composition in Japan, 4th revised edition. 4In the questions regarding coffee and black tea, if sugar and milk were taken for drinking either beverage, we added 4 g for the sugar, 50 g for the milk and 2.5 g for substitute cream, each of which is the median amount used per cup in WDR.Then we calculated the sum of the products of the frequency weight and the nutrient content of the portion size for each food as the daily nutrient intake.

Data Analysis
We tested the reproducibility of intake frequency from FFQs 1 & 2 and the amounts of the various nutrients calculated from FFQs 1 & 2 by the Spearman rank correlation coefficient between the two FFQs, which were carried out twice at the interval of one year.Because this is a multi-center research study, the standardization of the method was necessary.Therefore, those in charge of the dietary survey in every group met together before the research started to receive detailed instructions about the data coding system and the research procedures.Because both food and cooking names were given in the food item section in the questionnaire, we decided to code both of them in order to compute nutrient contents, recipe by recipe.
The subjects were provided with a digital cooking scale (TANI-TA No. 1141, Tokyo).After the research, the cooking scales were given to them as a token of our appreciation of their cooperation.

Data Collection
When the WDR was finished for each season, the person in charge of the dietary survey collected the recording sheets from WDR.Each group's dietitian codified food items and food weights.Those data were mailed out to the dietary survey center in Osaka, where the dietitian in the dietary survey center checked the contents of the input data.If she found something ambiguous or unprocessed, she contacted the person in charge for clarification, thus completing the processing of the appropriate data.The self-administered FFQs 1 & 2 were sent to Nagoya, the center of the JACC Study.After they were processed at Nagoya University, the FFQ data were mailed to the dietary survey center.

Computation of Nutrient Intakes from WDR
Based on the Standard Tables of Food Composition in Japan fourth revised edition, 4 and the fatty acid composition table of Japanese food, 5 we calculated the amounts of energy, protein (animal and vegetable), fat (animal, vegetable and fish), iron, calcium, potassium, salt, vitamin A, vitamin B1, vitamin B2, vitamin C, total fatty acid, saturated fatty acid (SFA), monounsaturated fatty acid (MUFA), polyunsaturated fatty acid (PUFA), and cholesterol.However, since the table has no entries for individual fatty acid components, we obtained a fatty acid composition database from Nagoya City University (Professor S. Tokudome) 6 and from the National Cancer Center Research Institute East (Dr.S. Sasaki). 7

Computation of Nutrient Intakes from FFQ
The content of FFQs used in the research are as follows: the number of bowls of steamed rice consumed daily at baseline; the intake frequency of miso soup; and the number of bowls a day at baseline; questions about the intake frequency of 33 kinds of food items, with 5 multiple choice responses: almost never; 1-2 times a month; 1-2 times a week; 3-4 times a week; almost everyday.There were other questions about the drinking habits of alcoholic beverages and soft drinks: coffee, black tea, Japanese green tea and Chinese tea.The questionnaires used in this study are shown in Appendix1 and Appendix2.
Each frequency weight was set at 0, 0.05 (1.5/30), 0.214 (1.5/7), and 0.5 (3.5/7), with 1.0 signifying the response 'almost everyday'.As for the frequency of miso soup, we set the four fre-Spearman correlation coefficients (both crude correlation coefficients, and energy adjusted correlation coefficients 9 ) between the WDR and the FFQ2.We used the software package (SPSS ® 11.5J for Windows, SPSS Inc.) for computation.

Analysis Subjects
Though unfortunately nine participants dropped out of the research, the remaining 85 persons completed the two FFQs (FFQ1 & FFQ2) and the 12-day WDR, and they were regarded as analysis subjects.Detailed information on them is shown in Table 2.

Validity of a FFQ used in JACC Study
In order to test the validity of the intake frequency of FFQ2, we defined the gold standard of intake frequency based on WDR as follows.First, corresponding to the frequency category of FFQ (almost never; 1-2 times a month; 1-2 times a week; 3-4 times a week; almost every day), we classified the frequency of 12-days WDR into five categories: never; 1 to 2 days; 3-5 days; 6-11 days; 12 days.Next, we counted the number of days during which the subjects ate each of the 33 food items, steamed rice, miso soup and beverages during the 12-day WDR and thereby determined the corresponding frequency categories.Then, we computed the Spearman rank correlation coefficients of 5 category classifications of the WDR and the FFQ2.
In order to examine the validity of the amount of nutrient intake computed from FFQ2 as mentioned above, we computed the Table 5 shows the percentages classified into the same category according to the two methods shown in Figure 2, the percentage classified into the same category and the contiguity category, and the percentage classified into the reversely categories, in the descending order within the same category.The mean percentage of those classified into the same category was 36%.Japanese green tea (59%) was the highest, whereas liver (any kinds) (5%) was the lowest.The classifications into reversely categories were found to exist in Japanese green tea (6%), Chinese tea (2%), milk, cheese, tofu, sweets, Chinese cabbage, and boiled beans (1%).

Nutrient Intake Assessment
Table 6 shows mean total energy and 11 nutrients intakes assessed with WDR and FFQ.Also, Table 7 shows the amount of fatty acid intake assessed with WDR and FFQ.The intake amount of energy and nutrients assessed with FFQ was lower than with WDR.The percentage difference{ (FFQ−WDR) / WDR } changed between −64% of salt and −15% of vitamin A. The mean percentage difference was −37% and, the median was − 36% among nutrients.
Table 8 shows the Spearman correlation coefficients between WDR and FFQ.We calculated 2 kinds of coefficients, using crude nutrient intakes and energy adjusted nutrient intakes by the

Intake Frequency Assessment
In Table 3, the Spearman rank correlation coefficients between FFQ1 & FFQ2 are shown, in the order of height.Coffee had the highest correlation coefficient (0.86), whereas edible wild plants the lowest (0.42).The correlation coefficients for all the foods were found to be statistically significant, and the mean and the median of 39 food items were 0.60.
In Table 4, the Spearman rank correlation coefficients between FFQ2 and 12-day WDR are shown, in the order of height.Coffee was found to have the highest (0.81) correlation coefficient.while edible wild plants the lowest (0.07).This order was the same as that of reproducibility.The mean was 0.39 and the median was 0.42.The foods whose correlation coefficients were not significant were deep-fried foods or tempura, preserved foods using soy sauce, kamaboko (fish paste), fried vegetables, and edible wild plants.
Figure 2 shows the relation between the frequency of Japanese green tea consumption and liver reported in the FFQ2, and their frequency evaluated in the 12-day WDR.We show the results by dividing them into three frames: the one where the frequency categories of the two methods corresponded, the one where the frequency categories are adjacent to each other, and the one where the frequency categories were in reversely results.

S-14
FFQ; food frequency questionnaire WDR; weighed dietary record  first examined the reproducibility and validity of intake frequency in detail, because frequency has a great influence on assessing the amount of nutrient intakes gained from FFQ. 2 The mean of the percentage classified into the same category was 36%, in which the intake frequencies assessed from WDR and FFQ.When the adjoining category was included, the mean value of the percentage of coincidence was 83%.In classifying the subjects according to intake frequency, it was shown clearly, making allowance for the misclassification by the adjoining category, that most of the food items were suitable for making the category evaluation of the subjects correctly.
Although the percentage in which Japanese green tea can be classified into the same category was the highest, that of reversal misclassification was the highest as 6% (5 subjects as shown in Figure 2).Two subjects out of the five subjects were classified as "never drinking" in WDR, whereas it is classified as "drink almost every day" in FFQ.The reason this discrepancy occurred was obviously that some of those who were drinking barley tea (mugicha) in WDR, could not make a distinction between barley tea and Japanese green tea in the FFQ, and so they chose the latter beverage, thus making the percentage of green tea higher than it really was.
The percentage of coincidence of liver (any kinds) in the same category was the lowest.We thought that this was probably because most of the subjects were estimated as "never" in the category assessed by 12-day WDR, in view of the fact that people rarely eat liver daily.Even if a classification also included the adjoining category, the percentage of coincidence of the frequency between WDR and FFQ was the lowest.
As mentioned above, except for special foods, we considered the validity of intake frequency good.As for reproducibility, the correlation coefficient with the range of 0.42-0.80 was shown by this study.
We already explained that FFQ examined in this study did not show the comprehensive amount of nutrient intake because the number of question items was 40.The daily amount of energy and nutrient intake from FFQ were lower in all than those from WDR.When the difference of the amount of intake between FFQ and WDR was showed in terms of its proportion to the amount of intake from WDR, the smallest value was 15 (vitamin A) whereas the largest value was 64 (salt).We considered that the reason the difference in vitamin A was smaller than that of the other nutrients was probably because the FFQ contained three items of green and yellow vegetables, which reflect their important role as a source of vitamin A. The FFQ did not contain any seasonings as food items.According to the national nutrition survey in Japan 10 , Japanese people obtain about 70% of the daily salt intake from soy sauce, miso and such seasonings.All this explains why the difference in the amount of salt intake between FFQ and WDR was very large.Also, it is usually very difficult to evaluate the amount of seasoning intake from a FFQ. 2,11sugane and others made a detailed validity research of 138item semi quantitative food frequency questionnaire used in Japan residual method. 8The Spearman correlation coefficient changed from 0.17 (linoleic acid) to 0.46 (animal fat, potassium), and the mean and median was 0.30 and 0.28, respectively.When energy adjusted nutrient intakes were used, the Spearman coefficient ranged from 0.15 (PUFA) to 0.51 (animal fat).The mean and median was 0.33 and 0.35, respectively.
In an epidemiologic study, researchers usually use evaluation by categorical classification rather than by the absolute value.We examined the validity of evaluating the amount of nutrient intakes by FFQ in four categories by quartiles.The percentage of coincidence (same category) varied between 45% (potassium) and 22% (MUFA).The mean coincidence percentage was 31% and the median was 33 %, and both the mean and median of the reversal category were 7%, and varied between 2% (potassium) and 11% (energy).

DISCUSSION
In order to study the relation between chronic diseases and dietary factors, a researcher should investigate a subject's habitual dietary intake over a long period of time, rather than during a short term.The food frequency questionnaire method (FFQ) has been developed to show habitual dietary intake. 2 If this method is used, it will, first of all, require a validity test to verify whether it is actually capable of showing an individual's habitual intake condition.A gold standard will be required for its validity test.We adopted the weighed dietary recording method (WDR) as the gold standard.The subjects recorded their dietary intake for 3 days every three months, totally 12 days in one year.In doing so, we took into consideration both their day-to-day variation and seasonal variation in dietary intake.Strictly speaking, subjects participating in the validity research should be chosen at random from cohort members belonging to the JACC Study.However, since WDR would be a considerable burden in terms of time and otherwise, we had to depend on volunteer members who understood the main objective of the research.Hence, the research results may not necessarily reflect the whole picture of the JACC members.
Because JACC Study was a large-scale cohort study involving more than 110,000 people, a self-administered food frequency questionnaire was used.Because the main purpose of the JACC Study was to clarify the risk factors of cancer, the questionnaire covered various areas, part of which contained questions about dietary intake.The questions were simple, mostly concerning intake frequency of a small number of food items, not their portion sizes.Second, because the focus of the questionnaire was on the kinds of food which had been considered to be the risk factors of cancer in the previous studies, 9 we failed to evaluate the comprehensive situation regarding nutrient intake.Third, although there are ordinarily 100-150 food items are needed to evaluate the amount of nutrient intake comprehensively 2 , the research this time had only 40 items.
Considering the limited nature of the questionnaire above, we Public Health Center-based Prospective Study on Cancer and Cardiovascular Diseases (JPCH Study), and published a series of papers on their research results. 12The JPCH Study was a cohort study targeted at residents of the areas under the jurisdiction of 4 public health centers in Akita Prefecture, Iwate Prefecture, Nagano Prefecture, and Okinawa Prefecture.They chose subjects for the validity study from cohort members as volunteers.For the gold standard, they used 7-day semi-weighed dietary records four times a year (winter, spring, summer and autumn, and a total of 28 days) or 7-day dietary records two times a year (winter and spring and a total of 14 days).Sasaki obtained 102 men and 113 women as subjects and made a report about them by the sex.
Our research method was different from the JPCH validation study in that our dietary records as the gold standard were shorter and that our FFQ had only 40 items.However, our research design was similar to theirs in using the Standard Tables of Food Composition in Japan, 4th revised edition 4 in computing nutrients.Because most of the targeted subjects were females in our study, we compared them with those in the JPCH Study.The JPCH classification was made into 5 categories by quintile, while ours was in 4 categories by quartile.The JPCH Study, which used 138 items in their FFQ, was capable of evaluating dietary intakes comprehensively.On the other hand, the correlation coefficients and analysis based on joint classification in the JACC Study were slightly lower than those of the JPCH Study.In determining portion sizes of food items, we adopted the median values of WDR, rather than general standard numerical values.This might have somehow raised the correlation coefficients between FFQ and WDR.
In conclusion, we found our FFQ suitable to deal with a large group of subjects.However, since the energy and the amount of nutrient intake from this FFQ can not show the comprehensive situation of dietary intake, subjects' dietary intake should be assessed by categories.

Figure2.
Figure2.Contingency tables for joint classification intake frequency assessed by 2nd food frequency questionnaire and weighed dietary record for 12 days.

Table 1 .
Portion sizes for 33 food items.

Table 2 -
2. Age distribution of analysis subjects.

Table 2 - 1 .
The number of analysis subjects by group and sex.

Table 5 .
Comparison of 2nd food frequency questionnaire with weighed dietary record for 12 days for frequency on joint classification by five categories.(%)

Table 6 .
Energy and nutrient intakes assessed with weighed dietary record (WDR) for 12-days and food frequency questionnaire (FFQ).

Table 7 .
Fatty acid intake assessed with weighed dietary record (WDR) for 12-days and food frequency questionnaire (FFQ).