2023 Volume 261 Issue 3 Pages 249-256
Web-based post-bereavement survey systems for specialized palliative care will enable obtaining timely results on the care quality from more participants at a lower cost. The primary aim of the study was to develop a web-based post-bereavement survey system and to compare response rates for different number of items. The secondary aim was to examine response bias between web-based and mail survey in post-bereavement surveys. Between January and April 2019, two cross-sectional web-based questionnaire surveys were conducted among the bereaved families from six inpatient palliative care units in Japan. Measurements included structure and process of end-of-life (EOL) care, overall care satisfaction, achievement of a good death, depression, grief status, web survey usability, and participant and bereaved family member characteristics. The long survey included 34 items, and the short survey included 16 items. There were no significant differences in the response rates between the long and short surveys (24% and 27%, respectively, p = 0.376). Compared with a previous nationwide post-bereavement mail survey, more children responded; however, the quality rating scores was unchanged. Despite low response rate, no apparent response bias was observed, indicating its feasibility. This survey method is low-cost, less burdensome to the institution, and allows for ongoing quality assurance.
Quality management of hospice and palliative care should preferably occur at national, regional, and institutional levels as well as during its planning and implementation (World Health Organization 2021). To this end, post-bereavement quality assurance surveys have been conducted in hospice and palliative care areas over the last few decades (Addington-Hall and McCarthy 1995; Lynn et al. 1997; Teno et al. 2004; Lawson et al. 2013; Hunt et al. 2019). The main purpose of post-bereavement surveys is to assess end-of-life (EOL) care provided, identify issues, and make improvements to manage the quality of hospice and palliative care. In the Japan Hospice Palliative Care Studies (J-HOPE studies), 10,000-scale bereaved family mailed surveys have been conducted for national quality management of specialized palliative care services (Miyashita et al. 2008b; Aoyama et al. 2017). The J-HOPE study had three objectives: 1) to evaluate the processes, structures, and outcomes of palliative care in hospitals, palliative care units (PCUs), and home hospice services; 2) to examine bereaved family members’ self-reported psychosocial conditions; and 3) to provide data to participating facilities to improve their care. However, this study had several limitations. First, most participating facilities were limited to PCUs. Second, because the survey was conducted once every four years, we could not obtain data in a timely manner. Finally, mail surveys are expensive to conduct. Recently, the Japanese government initiated a nationwide mortality follow-up survey based on death certification, allowing for a more representative survey (Nakazawa et al 2021). The J-HOPE study will be conducted only in PCUs.
To overcome the two problems of timeliness and cost, Hospice Palliative Care Japan (HPCJ) started developing a web-survey system for post-bereavement. The HPCJ is a specified nonprofit organization that aims to contribute to the quality of specialized palliative care, education, and dissemination of palliative care, with 85% of PCUs participating. The advantages of a web-based post-bereavement survey are as follows: 1) the system can be developed with 1/6 of the budget of the J-HOPE mail survey, which has a budget of approximately 16 million yen per survey, and the annual maintenance cost is meager; 2) facilities using the system can continue to conduct surveys and receive immediate feedback from the bereaved families; and 3) the number of facilities can be increased at a low cost. However, low response rates and response bias are expected for web-based post-bereavement surveys depending on the number of questions (Galesic and Bosnjak 2009).
The primary aim of the study was to develop a web-based post-bereavement survey system and to compare response rates for different number of items. We hypothesized that a smaller number of items would increase the response rate. The secondary aim was to examine response bias between web-based and mail surveys in post-bereavement survey.
A web-survey system was developed for this project. This system was created in collaboration with Hokuto Corporation, which has experience in web-based surveys. This system consists of three parts: 1) the HPCJ administrator, 2) the institutional administrator, and 3) the respondent. In the HPCJ administrator part, the following are possible: registration of the institution, creation of an ID and password for institutions to log into the system, uploading questionnaires, confirming data collected from all the participating institutions, and data download. The following are possible in the institution administrator part: registration of deceased patients, including patient demographic data, and printing out invitation letters for bereaved families, including URL/QR code of the log on the page. In the respondent section, the participants logged in and answered questions online.
Participants and proceduresTwo cross-sectional web-based surveys were conducted using convenient samples from six PCUs with experience in the J-HOPE study. The first survey was a long survey with 34 questions, conducted between January and February 2019. The second survey, a short survey with 16 items, was conducted between March and April 2019. The 16 items in the short survey were carefully selected from the 34 items in the long survey that were considered more important based on the discussion among the researchers. Both potential participants were bereaved family members of 100 consecutive patients who died between July 1, 2017, and June 30, 2018, meeting the following inclusion criteria: 1) the patient died from cancer, and 2) the bereaved were at least 20 years of age. The exclusion criteria were as follows: 1) the bereaved had psychological distress at the level that kept them from participating in the study, as determined by the primary physician, and 2) the bereaved were incapable of completing the questionnaires due to cognitive dysfunction or inability to read the Japanese language.
We asked each participating institution to divide the participants into two halves by the period of death allocated to the two surveys. The participating institutions sent invitation letters to primary caregivers by mail to the bereaved family members and asked them to respond to the questions.
This study was approved by the institutional review board in the Tokyo Women’s Medical University (approval No. 4863) and by the institutional review boards in each institution.
The sample size was based on the assumption that the response rate was lower than that in the J-HOPE mail survey: 15% for long and 25% for short surveys. This assumption resulted in a sample size of 270 respondents for each survey with 80% power. Each PCU included approximately 50 respondents, for a total of approximately 300 respondents per survey.
MeasurementsCare Evaluation Scale version 2.0, short version (CES2.0, short version): The short version of the Care Evaluation Scale, version 2.0, is used to measure EOL care from the perspective of bereaved family members, especially focusing on the structure and process of care (Morita et al. 2004). The scale consists of 10 items scored on a 6-point Likert scale (“1 = highly disagree” to “6 = highly agree”). There is “7: N/A” if none of the other scores is applicable. In this study, we used a short version consisting of all ten items from each subscale for the long survey, and we selected five items from the short version for the short survey.
Overall care satisfaction: We asked about their overall satisfaction with care using the following question: ‘Overall, in the past month, were you satisfied with the medical care the patient received in the last place of care?’ Participants were asked to respond on a 6-point Likert scale (“1 = highly dissatisfied” to “6 = highly satisfied”) (Morita et al. 2004).
Good Death Inventory, short version (GDI, short version): The short version of the GDI evaluates EOL care from the perspective of bereaved family members (Miyashita et al. 2008a). The short version consists of 10 cores and eight optional items. Each item was answered in six stages: “1: absolutely disagree” to “6: absolutely agree.” In this study, 10 items of the shortened version were used for the long survey, and four of the ten items of the shortened version were used for the short survey.
Patient Health Questionnaire-2 (PHQ-2): The PHQ-2 is a two-item version of the nine-item Patient Health Questionnaire depression module (PHQ-9) (Kroenke et al 2003). The scale inquiries about the frequency of depressed mood and anhedonia over the past two weeks, scoring each as “0: not at all” to “3: nearly every day” (Muramatsu et al 2018).
Brief Grief Questionnaire (BGQ): The BGQ was used to measure each participant’s grief (Shear et al 2006). It comprises five items rated on a three-point Likert scale (Ito et al. 2012). In this study, we administered two items to the long survey and one to the short survey.
Free comments for received care in the PCU: We asked the participants to write free comments on the care received in the PCU.
Feasibility of the web-survey: To investigate the usability of the system, we originally developed two items based on discussions among researchers, which were both rated on 6-point Likert scales: “how was the ease of use of this web-survey system?” and “did you find something good by answering this web-survey?” For the latter item, if the answer was “very good,” “good,” or “somewhat good,” we asked for the reasons.
Participants and bereaved family member characteristics: Information on the patients’ age, sex, primary cancer site, admission date in the PCU, and date of death was collected from both surveys by inputting the information into the system by each institution. Data on family age, sex, relationship with the deceased, physical and mental health status during the caregiving period, and frequency of patient attendance were collected through a web-survey.
AnalysisWe calculated the descriptive statistics for all items. To examine if there was a difference between the long and short surveys, we compared the participants’ and respondents’ characteristics, CES, GDI, PHQ-2, BGQ, and feasibility of the web-survey using the chi-square test, Student’s t-test, Wilcoxon rank-sum test, or Fisher’s exact test, where appropriate. The results of J-HOPE3 mail survey are shown with the result of the long and short surveys in the table to examine bias. The J-HOPE3 mail survey consisted of a core part of 58 items and an additional part with different question content and number of items for each subject. We also conducted a theme and content analysis of these comments. We set the significance level to p = 0.05 using a two-tailed test. All statistical analyses were conducted using IBM SPSS Statistics 21 or R for Windows version 4.1.2.
For the long survey, 268 invitation letters were mailed, and 64 participants responded to the web-survey (24%). For the short survey, 274 invitation letters were mailed, and 74 participants responded to the web-survey (27%). There were no significant differences in the response rates between the long and short surveys (p = 0.376) (Table 1).
Comparison of respondents’ and non-respondents/refusals’ characteristics revealed that the response rate was not significant except more female in the long survey (p = 0.017) and longer hospital stays in the short survey (p = 0.005) (Table 2).
Response rate.
Comparison of characteristics in respondents and non-responders/refusals.
SD, standard deviation; PCU, palliative care unit.
No differences in patient characteristics were observed between the long and short surveys, except for patient age (74.5 vs. 70.2 years, p = 0.04) (Table 3). In this survey, more children answered than in the J-HOPE3 mail survey (long survey, 59%; short survey, 39%; and J-HOPE3, 38%).
Participants’ characteristics.
Some totals are not 100% due to missing values.
The results of J-HOPE3 are shown so that the difference between the results of this web survey and the mail survey can be seen.
aChi-square test, bStuden’s t-t test, cWilcoxon rank-sum test, dFisher’s exact test.
SD, standard deviation; PCU, palliative care unit; J-HOPE3, Japan Hospice and Palliative Care Evaluation Study 3.
There were no significant differences between the long and short surveys in CES, overall care satisfaction, GDI, PHQ-9, and BGQ scores (Table 4). Comparing the web long survey and J-HOPE3 mail survey, ranges of mean CES scores (SD) were 4.7-5.2 (1.0-1.3) and 4.7-5.3 (0.7-1.2), respectively. Mean Overall care satisfaction scores were 5.1 (1.0) and 5.0 (1.0), respectively. Ranges of mean GDI scores were 3.0-6.2 (1.1-2.0) and 3.0-6.0 (1.0-1.9), respectively. Mean PHQ-9 scores were 0.7-0.8 (0.9) and 0.7 (0.8-0.9), respectively. Mean BGQ scores were 1.1-1.2 (0.6) and 1.0-1.4 (0.6-0.7), respectively (Table 4).
The distributions of outcomes.
The results of J-HOPE3 are shown so that the difference between the results of this web survey and the mail survey can be seen.
CES, Care Evaluation Scale ver.2. The scores range from 0 to 6, with a higher score representing a higher level of agreement.
Overall satisfaction ranges from 0 to 6, with a higher score representing a higher level of satisfaction.
GDI, Good Death Inventory. The scores range from 0 to 7, with a higher score representing a higher level of agreement.
PHQ-9, the Patient Health Questionnarie-9. The score ranges from 0 to 4, with a higher score representing a higher level of trouble.
BGQ, the Brief Grief Questionnaire. The score ranges from 0 to 3, with a higher score representing a higher level of grief.
SD, standard deviation; PCU, palliative care unit; J-HOPE3, Japan Hospice and Palliative Care Evaluation Study 3.
Regarding the ease of using this web-based survey system, most respondents answered from “very easy to use” to “somewhat easy to use” (64% vs. 76%, p = 0.50). For the question of benefits answering this web-survey, most respondents answered from “very good” to “a little good” (75% vs. 89%, p = 0.27) (Table 5).
Recognitions of using the web-survey.
Fisher’s exact test was used for all tests.
Of the 46 comments, 67% and 70% were thankful words, and 45% and 22% were opinions/complaints in the long and short surveys, respectively. The median number (range) of written words was 103 (7-723) words and 92 (9-616) words, respectively.
A significant finding of this pilot study is the identification of response rates for the web-based bereavement survey. The response rate for both surveys was approximately 30%. They were higher than expected but lower than the J-HOPE’s response rate of about 70% for the self-administered questionnaire mail survey (Miyashita et al. 2008b, 2015; Aoyama et al. 2017). A possible reason is that the potential survey participants were older and less familiar with the web-survey. The response rate for this survey was lower than that of Zhao’s servey (43.6%) (Zhao et al. 2021). As Zhao’s servey was administered only to those who responded to the preliminary survey, we did not know what the response rate would have been if the preliminary survey had not been conducted.
Differences in the number of items did not affect the response rate or the respondents’ background. This rejects our hypothesis that a smaller number of items would increase the response rate. This might be because numbers 34 and 16 were not significantly different from the response, especially in the online survey. If the response rate did not change, it would be reasonable to use a longer survey to obtain more information.
In both surveys, more than 60% of respondents offered a positive evaluation of the usability of the web-based survey system. In the free description, 70% of the responses were words of gratitude to the facility. It would be encouraging for the staff to receive direct gratitude from the bereaved family in a setting where the patient will eventually die. These results show that the web-based survey system was satisfactory. If the response rate did not change, it would be reasonable to use a longer survey to obtain more information.
In this survey, more children answered than in the J-HOPE3 mail survey (long survey, 59%; short survey, 39%; and J-HOPE3, 38%). The child might have input the data because the parent (husband/wife of the deceased) cannot use the Internet and ask the child to input data with the parent. The distributions of the CES, GDI, PHQ, and BGQ items were similar between the web-based surveys and the J-HOPE3. In a comparison between respondents and non-respondents, the shorter the length of hospital stay, the lower the response rate. This was similar to the J-HOPE mail survey (Miyashita 2016). These results indicate no specific biases in the responses because this was a web-based survey.
This study had some limitations. First, the number of participating facilities and participants was small and may have been biased. Second, in the interviews with each participating facility, after the survey was completed, opinions on system improvement were provided, although all were minor points. In addition to improving them, the challenge is to improve and evaluate the system so that it will be more accessible and more beneficial for the facilities, such as setting evaluation items for each facility and setting the aggregation period.
In conclusion, we developed a web-based post-bereavement survey system for EOL care evaluation and examined its feasibility in a pilot study. Although the response rate of this survey was low, no apparent response bias was observed, indicating its feasibility. This survey method is low cost, less burdensome to the institution, and allows for ongoing quality assurance. In the future, improving the system, expanding the number of targets and settings, and evaluating the system by the facility will be an issue.
This work was supported by JSPS KAKENHI Grant Number JP19K11174. The authors express their gratitude to the bereaved family members who participated in the study, the hospital managers who implemented this study, and the hospitals who permitted this study.
T.S. substantially contributed to the study conceptualization and drafted the original manuscript. K.S. significantly contributed to the interpretation of the results. A.H., A.S., S.O., Y.S., Y.S. and T.H. substantially contributed to the data collection and interpretation. Y.S. supervised the conduct of this study. M.M. supervised the conduct of this study and secured funding. All authors critically reviewed and revised the manuscript draft and approved the final version for submission.
The authors declare no conflict of interest.
