2023 年 23 巻 1 号 p. 18-32
In our past cohort study, we carried out nationwide web-based surveys to assess mental health deterioration among non-permanent workers over a 1-year period during the COVID-19 pandemic in Japan. The association between non-permanent employment and adverse mental health conditions was found to be the basic characteristic of the original sample. However, high non-response rates and selective drop-out occurred, suggesting that the samples in the follow-up surveys might have been biased. The aim of the present study was to describe changes in sample characteristics among our three surveys and factors associated with non-response in the third survey using data at baseline and the second survey. Our web-based survey of the working-age population in Japan included three waves: March 26 to April 6, 2020; June 26 to July 2, 2020; and October 26 to November 5, 2021. From the original cohort of 3001 Japanese workers, 2351 and 1568 respondents participated in the second and third surveys, respectively. The proportions among respondents of young, single, and non-permanent workers decreased across the three surveys, as did Center for Epidemiological Studies Depression Scale (CES-D) and sense of coherence (SOC) scores; however, EuroQol 5-dimension 5-level (EQ-5D-5L) scores increased across surveys. No differences were found in CES-D, SOC, and EQ-5D-5L scores between non-permanent and permanent workers among respondents in the third survey before and after propensity score matching. The risk factors associated with non-response in the third survey were female sex, younger age at baseline and lower EQ-5D-5L scores in the second survey. The basic characteristics of the sample changed owing to non-response bias, resulting in underestimation of mental health deterioration among non-permanent workers in our cohort study. There is a need for additional cross-sectional studies to complement the assessment of adverse mental health symptoms among workers in Japan at a given time point during the COVID-19 pandemic.
We conducted a cohort study comprising nationwide web-based surveys to assess mental health among Japanese workers during the COVID-19 pandemic [1-3]. However, a main concern is that only having results from repeat respondents may lead to overly optimistic interpretations of mental health trends. Our previous study showed that the prevalence of depression increased by approximately 52% before the declaration of the COVID-19 state of emergency in Japan [1,2], but this prevalence decreased after the emergency declaration [3]. We also revealed that non-permanent workers had poorer mental health status than permanent workers [2], in line with findings of another nationwide survey conducted before the COVID-19 outbreak in Japan [4]. The association between non-permanent employment and adverse mental health conditions was the basic characteristic of our original sample, which can be generalized directly to the Japanese general population. However, the change in depressive symptoms was not significantly different between permanent and non-permanent workers in Japan after the first state of emergency owing to COVID-19 [3].
In our previous study, we had a high non-response rate and selective drop-out in the follow-up survey before and after the COVID-19 state of emergency declaration in Japan [3, 5]. In the follow-up survey, we reported a non-response rate of approximately 22%. Furthermore, the risk factors associated with non-response in our previous cohort study were younger age, low income level, being single, not being the head of household, and low sense of coherence (SOC) [5]. Respondents with the characteristics of socially vulnerable individuals selectively dropped out from the initial cohort sample [5]. The most important consequence of the high non-response rate is bias in the survey estimates caused by selective drop-out [6]. Thus, although intragroup comparisons over time are a key strength of longitudinal cohort studies, a main threat to the validity of results from such studies is non-response in follow-up surveys [7].
The bias introduced owing to selective drop-out could result from restricting the analytic sample to respondents who consistently participated in the surveys, ignoring individuals who dropped out of the cohort study [8]. When only a subset of all participants provides follow-up information on exposures and outcomes, the participating subset may not be representative of the original sample [6]. The basic characteristics of the original sample could change during the follow-up period with a high response rate and selective drop-out, suggesting underestimation of the adverse effects of the COVID-19 state of emergency on mental health among non-permanent workers in Japan [5]. Therefore, it is important to consider the representativeness of the follow-up sample in interpreting the results of the cohort study.
In our cohort study, it is unclear whether non-response bias caused a change in baseline characteristics and resulted in underestimated mental health deterioration among non-permanent workers during the COVID-19 pandemic in Japan. The aim of the present study was to describe changes in characteristics of the samples across our three surveys and factors associated with non-response in the third survey over a follow-up period of 1 year during the COVID-19 pandemic in Japan. Mental health characteristics included in the present study model are similar to those in our previous studies [1-3, 5]. We examined differences in the characteristics of respondents to the three surveys. We then examined the differences in characteristics between respondents and non-respondents to the third survey using data at baseline and in the second survey.
1. Study design and data collection
We conducted a nationwide survey of Japanese workers aged 15-59 years. Data were collected via an online platform provided by a research company (Cross Marketing Corporation, Tokyo, Japan). We ensured representation of the Japanese population in terms of residential area, age, and sex during data collection. In total, 3001 responses from all 47 prefectures were collected in the first survey from March 26 to April 6, 2020. The respondents provided explicit consent to participate, and all survey data were completely anonymous. In the second survey conducted from June 26 to July 2, 2020, all participants in the first survey were invited to participate. After 1 year, all respondents in the second survey were invited to complete the third survey during the period from October 26 to November 5, 2021. From a cohort of 3001 Japanese workers, there were 2351 (78.3%) and 1568 (66.7%) respondents and 650 (21.7%) and 783 (33.3%) non-respondents in the second and third surveys, respectively. Figure 1 shows the design of the present study. We determined the differences in the sociodemographic and mental health characteristics of respondents between the follow-up samples and the original sample in the three surveys. We then compared the characteristics between respondents and non-respondents in the third survey using data at baseline and the second survey.
2. Demographic variables and health at baseline
1) Employment status
We defined four categories of employment status. First, permanent workers included general employees, managers, or officers who were employed full-time. Second, non-permanent workers included those with a fixed-term employment contract as well as temporary and part-time workers. Third, civil servants comprised public service workers in national or local government. Fourth, self-employed workers referred to self-employed individuals, such as sole proprietors and freelancers. Survey respondents’ employment status was self-reported.
Among the 3001 workers in the first survey, we excluded data for housewives, students, and unemployed people. However, we investigated unemployment during the COVID-19 pandemic in the second and third surveys.
2) Socioeconomic status and outcomes
Sociodemographic factors included age, sex, marital status, personal income, and head of household status.
We used the Center for Epidemiological Studies Depression Scale (CES-D) to measure depressive symptoms, the sense of coherence (SOC) scale to assess stress-coping ability, and the EuroQol 5-dimension 5-level (EQ-5D-5L) questionnaire to assess health-related quality of life (HRQOL).
The CES-D comprises 20 items in which participants are asked to rate how often they have experienced symptoms associated with depression over the previous week [9]. We used the Japanese version of the CES-D. Responses are on a four-point rating scale, with a final score of 0-60 [10]. We used a score of ≥16 points as the cutoff to indicate depressive symptoms [11].
We assessed stress-coping ability using the Japanese version of the SOC scale, with higher scores indicating better stress-coping ability [12, 13]. This scale has 13 items and responses are on a seven-point rating scale, with a total possible score of 13-91. We divided participants into three groups based on their SOC score: low (scores 13-45), medium (46-59), and high SOC (60-91) [14].
Next, we used the Japanese version of the EQ-5D-5L to assess respondents’ HRQOL [15]. The EQ-5D comprises five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Each domain is rated on a scale that describes the degree of problems in that area (i.e., “I have no problems walking about,” “slight problems,” “moderate problems,” “severe problems,” or “unable to walk”). Scores range from −0.025 to 1, with 0 indicating death and 1 indicating perfect health in the Japanese value set (a negative score indicates a health status worse than death).
3. Statistical analyses
Tukey’s multiple comparison test was used to compare the mean CES-D, SOC, and EQ-5D-5L scores according to employment status among respondents in the third survey. Propensity score matching (PSM) is to make the selected permanent and non-permanent workers groups among respondents in the third survey comparable in terms of potential confounding factors, in order to balance variables and reduce bias [16]. To calculate propensity scores, we conducted logistic regression with baseline socioeconomic factors. The permanent workers group and the non-permanent workers group were paired 1:1 based on the propensity scores using the nearest-neighbor matching method with a caliper of 0.2 standard deviations. Then, single-factor analyses were used to compare the differences in CES-D, SOC, and EQ-5D-5L scores between the two groups.
Demographic characteristics were assessed using chi-square tests to assess differences between non-respondents and respondents in the third survey by sex, age group in years, head of household status, employment status, marital status, and personal income. Some data for CES-D, SOC, and EQ-5D-5L scores were non-normally distributed. Differences in mental health measures were assessed using the Mann-Whitney U test, a non-parametric statistical method [17].
Logistic regression analyses were performed to examine the sociodemographic and mental health determinants of non-response in the third survey. Prevalence odds ratios (ORs) and 95% confidence intervals (CIs) for non-response were estimated using logistic regression analyses. The independent variables were socioeconomic factors, SOC group (low, medium, or high), depressive symptoms (CES-D score ≥16), and EQ-5D-5L score. Standard errors of the beta coefficients were calculated to avoid multicollinearity concerns in the multivariable analyses. All standard errors of the independent variables used in this study were less than 2.0 [18]. PSM analysis was performed using EZR software (Easy R version 1.27; Saitama Medical Center, Jichi Medical University, Omiya, Japan) and other analyses were conducted using BellCurve Excel 2016, version 4.02.
Table 1 shows changes in the characteristics of respondents across the three surveys. Regarding age, 17.7% of respondents aged ≤24 years participated in the first survey; this age group accounted for 10.3% and 7.4% in the second and third surveys, respectively. The percentage of workers who were single, non-permanent workers, and who had depressive symptoms decreased among respondents across the three surveys. Respondents in the third survey had the lowest CES-D and SOC scores and the highest EQ-5D-5L scores.
Table 2 describes the comparisons of CES-D, SOC, and EQ-5D-5L scores according to employment status among respondents in the third survey. There were no statistically significant differences in mean CES-D, SOC, and EQ-5D-5L scores between non-permanent workers and permanent workers, civil servants, or self-employed respondents. However, the mean CES-D score of unemployed people was significantly higher than that of non-permanent workers (p < 0.05). The mean SOC and EQ-5D-5L scores of unemployed people were significantly lower than those of non-permanent workers (p < 0.05).
The distribution of demographic characteristics between the permanent workers group (n = 908) and the non-permanent workers group (n = 425) is shown in Table 3. Before PSM, there were significant differences among the following factors: sex, personal income, and head of household status (p < 0.001). After PSM, the permanent workers group and the non-permanent workers group were paired 1:1, as described in the Materials and Methods section. As a result, there were demographic-characteristic-matched participants in the permanent workers group (n = 235) and the non-permanent workers group (n = 235) after PSM, and the differences in the above-mentioned demographic characteristics disappeared between these two groups (p > 0.05). After PSM, there were no differences in mean CES-D, SOC and EQ-5D-5L scores between the two groups (p > 0.05).
Table 4 presents the results of univariate analysis for non-respondents and respondents in the third survey. Notably, the characteristics of non-respondents and respondents (i.e., age, marital status, and HRQOL) differed significantly. Using characteristics at baseline, we found that non-respondents in the third survey were more often younger and single workers than respondents. Using characteristics from the second survey, we found that non-respondents in the third survey had lower EQ-5D-5L scores than respondents.
Logistic regression analyses revealed that the risk factors at baseline associated with non-response in the third survey were female sex (OR [95% CI]: 1.380 [1.130-1.690]) and younger age (OR [95% CI]: 2.840 [2.060-3.920]). Higher EQ-5D-5L scores in the second survey were inversely associated with non-response in the third survey, with OR [95% CI]: 0.499 [0.253-0.982] (Table 5). The fact that the 95% CI of ORs did not include 1.0 indicates that the coefficients of each item in the logistic regression formula were statistically significant.
|
|
|
|
|
|
|
|
To clarify the psychological impact on non-permanent workers during the COVID-19 pandemic in Japan, we conducted a cohort study over a 1-year period. However, the non-response rate was high and selective drop-out occurred in the follow-up survey [3, 5]. The basic characteristics of the original sample could have changed, suggesting underestimation of mental health deterioration among non-permanent workers during the COVID-19 state of emergency in Japan [5]. The aim of the present study was to describe changes in sample characteristics among our three surveys and factors associated with non-response in the third survey using data at baseline and the second survey. We found that proportions of young, single, and non-permanent workers as well as CES-D and SOC scores decreased and EQ-5D-5L scores increased among respondents across the three surveys. We found no differences in CES-D, SOC, and EQ-5D-5L scores between non-permanent and permanent workers among respondents in the third survey before and after PSM. The risk factors associated with non-response to the third survey were female sex, younger age at baseline, and lower EQ-5D-5L scores in the second survey.
The proportions of young, single, and non-permanent workers decreased among respondents across the three surveys. Previous studies of health trends over time have reported similar findings [8, 19]. It is difficult to maintain consistency in a survey sample over long periods of time [19]. The follow-up sample represents a subset of respondents to the baseline survey owing to non-response bias, leading to results that cannot be generalized directly to the whole population [20]. Although we ensured adequate representation of the Japanese population in the initial survey, the general characteristics of the follow-up samples differed from those of the original sample, suggesting that the resulting analytic samples may be non-representative in our cohort study.
CES-D and SOC scores decreased and EQ-5D-5L scores increased among respondents across the three surveys. Depressive symptoms declined among respondents in the three surveys. Monitoring population trends of health indicators over time is essential to promoting health and preventing disease as well as planning for health care resources [21]. However, when the features of survey respondents change, it may be difficult to determine whether the changes are real or the result of changes in the representativeness of the sample [21]. Additionally, our previous study using data at baseline found that workers with lower SOC scores had greater depressive symptoms [1, 2], but no correlation was observed between SOC and CES-D scores in the third survey (data not shown). The declines in both the CES-D and SOC scores might be owing to changes in the characteristics of the samples in the three surveys.
Notably, no differences were found in CES-D, SOC, and EQ-5D-5L scores between non-permanent and permanent workers among respondents in the third survey. Nevertheless, our previous study using data from the initial survey showed that non-permanent workers had higher CES-D scores than permanent workers in the early stages of the COVID-19 pandemic [2]. Non-permanent workers reported greater depressive symptoms than permanent workers, which is consistent with a report by Inoue et al. in a nationwide survey conducted before the start of the COVID-19 pandemic in Japan [4]. The mental health characteristics of our original sample could be nationally representative. However, our follow-up surveys reported high non-response rates [3, 5]. Davern suggested that a response rate of 81% will not lead to non-response bias [26]. Gustavson et al. suggested that when individuals with extreme levels of health problems are not successfully retained in the sample, bias may be more severe than when the most troubled individuals are under-represented, but present in the sample [27]. PSM is a strategy to reduce the selection bias due to selective drop-out in cohort studies [16]. After PSM, the basic characteristics were maintained in the follow-up sample of the second survey [3], however, we could not find the basic characteristics of our original sample among respondents in the third survey.
Among respondents to our third survey, unemployed people showed the worst mental health status. Unemployment is a factor that negatively impacts mental health among Japanese workers [3]. Oswald et al. noted that the mental health implications of the pandemic appear to be particularly salient for young adults (age 18-24 years), who are more likely to have non-permanent employment and are at greater risk of job loss in a socioeconomic crisis like the pandemic [22]. Furthermore, non-permanent workers reported higher rates of unemployment than permanent workers in both the second [3] and third surveys (data not shown). However, there was an improvement in the CES-D and EQ-5D-5L scores in both two groups [3]. Additionally, workers with poor mental health conditions dropped out from our follow-up surveys [5]. It is reasonably suggested that non-permanent workers with poor mental health were selectively dropped out from our follow-up surveys, and the difference of mental health between non-permanent and permanent workers was lost among respondents in the third survey. Although non-permanent employment is an important social factor related to mental health deterioration in Japan [2], the adverse mental health effects during the COVID-19 pandemic among non-permanent workers might be underestimated in our cohort study.
As our study, several previous studies were conducted to find characteristics of non-respondents and risk factors of non-response in cohort studies [8, 23−25, 27]. Being young and single at baseline and having lower EQ-5D-5L scores in the second survey were notable characteristics of non-respondents in the third survey, which is consistent with our previous study [5]. Additionally, HRQOL and mental health are closely related, and the EQ-5D-5L includes the dimension of anxiety/depression. A previous study showed that scores in the physical and mental dimensions of HRQOL declined significantly from immediately before the COVID-19 outbreak to 1 year later in the Japanese general population [28]. Another previous study in Denmark reported that frequent use of social media appears to have a negative impact on mental health and QOL, and this is especially the case during the COVID-19 pandemic [29]. Hence, the negative impact on HRQOL as well as mental health during the pandemic might be associated with an increased non-response rate in follow-up among young workers who have high levels of social media use.
Female sex, younger age at baseline and lower EQ-5D-5L scores in the second survey were risk factors of non-response to the third survey, in line with our previous study [5] and other health surveys [25, 30, 31]. Some risk factors associated with non-response in the third survey were similar to those of non-response in the second survey [5]. Selective drop-out occurred repeatedly among workers who were younger and had lower EQ-5D-5L scores across our three surveys. Selective drop-out could pose a serious threat to the validity of findings regarding mental health trends among Japanese workers during the COVID-19 pandemic in our cohort study. Non-response will continue to be a great challenge in forthcoming surveys with a longitudinal design [7]. Therefore, conducting additional cross-sectional studies will be important to complement assessment of the prevalence of adverse mental health symptoms in Japan at a given time point during the COVID-19 pandemic [5, 8].
There are certain limitations that should be considered when interpreting our results. We were unable to collect further information from non-respondents in the follow-up surveys to clarify the exact reasons for non-response during the COVID-19 pandemic. We were unable to determine whether people did not respond owing to refusal or an inability to be contacted [6].
The association between non-permanent employment and adverse mental health conditions was a basic characteristic in the original sample of our cohort study. We revealed changes in sample characteristics and factors associated with non-response to the third survey using data at baseline and the second survey. No differences were found in mental health and HRQOL between non-permanent and permanent workers among respondents to the third survey before and after PSM. The risk factors of non-response to the third survey were female sex, younger age at baseline and lower EQ-5D-5L scores in the second survey. Non-response bias changed the basic characteristics of the follow-up samples, resulting in underestimation of mental health deterioration among non-permanent workers in our cohort study. Therefore, additional cross-sectional studies are needed to accurately assess adverse mental health symptoms among workers in Japan during the COVID-19 pandemic.
This study was supported in part by the Japan Society for the Promotion of Science KAKENHI (grant number 21 K10289). We thank Analisa Avila, MPH, ELS, of Edanz (https://jp.edanz.com/ac) for editing a draft of this manuscript.
The authors declare no conflicts of interest. This study was approved by the Medical Ethics Committee of Niigata University of Health and Welfare (No. 18385-200318; date: April 18, 2020) and was conducted in accordance with the principles of the 1964 Declaration of Helsinki and its later amendments.
