Evaluation of a Webinar for Pharmacists Learning Basic Clinical-Oncology during COVID-19 Pandemic in Japan

Abstract

It is essential for oncology pharmacists to update their knowledge, skills, and ethical attitudes. The Japanese Society of Pharmaceutical Oncology is an academic society for healthcare professionals involved in cancer treatment. It has conducted in-person seminars every year to cultivate the knowledge necessary for practicing advanced cancer medicine. Owing to the coronavirus disease (COVID-19) pandemic, the society was obligated to conduct a web-based seminar this year. A questionnaire survey was conducted before and after the webinar to explain how it works and to assess the learning attitudes of beginner and moderately skilled pharmacists in the field of oncology. Questionnaire surveys were conducted with the participants before and after watching the webinar. The questionnaires sought to determine participants’ perspectives on the webinar and their knowledge of the seven modules. Of the 1756 webinar attendees, 1661 (94.6%) answered the pre-webinar survey and 1586 (90.3%) answered the post-webinar survey. Results indicate that the median post-webinar knowledge score was significantly higher than the median pre-webinar score (p < 0.001) in all modules. Principal component analysis of the degree of knowledge of seven modules revealed that the improved score group consisted of those from younger age groups, with less experience as pharmacists, non-society members, and those with less experience in past society seminars. Moreover, the web-based seminar provided a uniform learning effect throughout the country without distinguishing between urban and rural learners. The web-based educational program was an acceptable educational tool for Japanese oncology pharmacists.

INTRODUCTION

The field of oncology is advancing every day, and it is essential for oncology pharmacists to update their knowledge, skills,¹⁾ and ethical attitudes.²⁾ As there is no renewal system for pharmacist licenses, the post-graduate education of pharmacists is dependent on their own desire and discretion. Cytotoxic anticancer drugs,^3,4) molecular targeted drugs,⁵⁾ and immune checkpoint inhibitors^6,7) cause a wide variety of side effects, and pharmacists are required to manage these side effects. Moreover, lung, breast, and colorectal cancers are the most common types of cancer, and there are many opportunities for pharmacists to intervene.^8,9) Palliative care is recommended from the early stages of the diagnosis of cancer, and opportunities for intervention are also present in palliative care.¹⁰⁾ Pharmacists need to acquire up-to-date knowledge in these areas in order to fully demonstrate their capabilities in team medicine. The Japanese Society of Pharmaceutical Oncology (JASPO) is an academic society for healthcare professionals involved in cancer treatment. JASPO has held in-person “start-up seminars” for beginner to moderately skilled pharmacists, “brush-up seminars” for intermediate to advanced pharmacists, and “expert seminars” to cultivate the knowledge necessary for practicing advanced cancer medicine. However, the pandemic made it impossible to conduct the event in the conventional format, forcing the organizers to switch to a remote learning style. The organization developed an on-demand program for remote learning to prepare pharmacists for the ongoing and advanced oncology pharmaceutical care training. It is unclear whether the webinar format will have the same learning effect as the conventional seminar format. The International Society of Oncology Pharmacy Practitioners (ISOPP), committed to providing educational resources to members for their continued learning and professional development, performed cross-sectional survey all over the world and reported that webinars were an effective educational tool to barriers such as lack of financial support, the great deal of time spent on travelling to attend educational activities, limited learning opportunities, and limited growth of the oncology pharmacy discipline.¹¹⁾ Furthermore, various reports revealed that e-learning program is effective for pharmacists’ learning.^12,13) Subsequently, we performed a questionnaire survey before and after the webinar to explain how it works and to assess the learning attitudes of beginner and moderately skilled pharmacists in the field of oncology.

MATERIALS AND METHODS

Webinar for Oncology Pharmacists

The educational webinar for beginner to moderately skilled pharmacists (JASPO Start-up seminar) was offered for seven days from November 30 to December 6, 2020. Participants could watch the webinars multiple times online and from any location. E-learning programs were originally presented in one day and were later expanded to seven days. The modules of the educational webinar covered the following topics: pharmaceutical care of cytotoxic anticancer drugs, molecular-targeted drugs, immune checkpoint inhibitors, colorectal cancer, lung cancer, breast cancer, and palliative care. Each 45-min program was pre-recorded by oncology pharmacy specialists in Japan.

Questionnaires

The questionnaires were developed referring a paper¹⁴⁾ and finalized with the agreement and understanding of all 15 committee members. Self-administered questionnaire surveys were given to participants before and after watching the educational webinar. The pre-webinar survey was accessible before watching the webinar, and the post-webinar survey was accessible after watching the webinar. Questionnaires were designed to determine the participants’ perspectives on the webinar, as well as the degree of knowledge of three types of drugs (cytotoxic, molecular-targeted, and immune checkpoint inhibitor), three types of cancer (lung, colorectal, and breast), and palliative care. Both before and after the webinar, the degree of knowledge was recorded on a ten-point scale, where 1 represented “I do not understand at all,” and 10 represented “I understand well enough.” We used the linkable anonymizing method to check responses to the questionnaires and compare the results of the pre- and post-webinar surveys. The perspectives of the web-based program were evaluated using a five-point Likert scale. The participants’ completion and return of the questionnaire were regarded as consent to participate in this research.

Degree of Knowledge before and after the Webinar

The knowledge scores for each of the seven modules making up both the questionnaires (before and after the webinar) were compared with Wilcoxon signed-rank test, and the distribution of score was represented by violin plot.

Principal Component Analysis (PCA) of the Degree of Knowledge of the Seven Modules

PCA performed before and after the scores of the seven modules were analyzed. To standardize the self-assessment, we used the pre-seminar values as a baseline and analyzed the change in the values. The respondents were divided into two groups based on the results of the PCA and we compared two groups regarding age, years of experience as a pharmacist, JASPO member or not, and experience of past JASPO seminars.

The Equalization of Self-evaluation between Urban and Rural Prefectures City

The differences in self-evaluation between the urban and the rural areas are explored in Table 1. We defined urban prefectures as those with a population of over 500000, and rural prefectures as those with a lesser population. The urban prefectures included Hokkaido, Miyagi, Saitama, Tokyo, Kanagawa, Niigata, Shizuoka, Aichi, Kyoto, Osaka, Hyogo, Okayama, Hiroshima, Fukuoka, and Kumamoto. Groups A and B are classified by the results of PCA analysis.

Table 1. The Difference of Self-evaluations between Urban and Rural Prefectures City

	Rural prefecture (<500000 people)		Urban prefecture (≥500000 people)		Pearson’s chi-square test
Improved or not	Group A	Group B	Group A	Group B	p = 0.82
Degree	135	128	362	332
Percentage (%)	14.1	13.3	37.8	34.7

Groups A and B are classified by result of PCA analysis. Group A was set as the less-improved group and group B as the improved group shown in Fig. 4-b.

Statistical Analysis

All statistical analyses were performed using JMP Pro 16 (SAS Institute Inc., Cary, NC, U.S.A.). The Wilcoxon signed-rank test was used to compare the scores of each module. PCA was performed before and after the scores of the seven modules were analyzed. Pearson’s chi-square test was performed to compare the differences in self-evaluation between the urban and the rural areas

Ethical Approval

This study was approved by the ethics review board of the Miyagi Cancer Center (Approval No. 2020-043).

RESULTS

Responses and Data

Of the 1756 webinar attendees, 1661 (94.6%) and 1586 (90.3%) answered the pre-webinar and post-webinar surveys, respectively. For responses where some questions were unanswered, we treated the blank data as “not applicable (N/A),” while retaining data gathered from other questions. We explored the respondents’ characteristics using the pre-webinar survey data (n = 1661) and their perspectives using the post-webinar data (n = 1586).

Characteristics of Respondents

The characteristics of the pre-webinar survey respondents are shown in Fig. 1. Most participants were aged between 25 and 29 years (28％), followed by those aged between 30 and 34 years (26％) (Fig. 1-a). Years of experience as a pharmacist were almost 1–30 years (Fig. 1-b). Over half of the participants were JASPO members (Fig. 1-c). Further, over half of the participants belonged to general hospitals (52%), and more than a third were placed at pharmacies (35%; Fig. 1-d). More than half of the participants (63%) had no experience of the past JASPO seminars (Fig. 1-e). Moreover, among respondents who had experienced past JASPO seminars, most had done so only once (56%) (Fig. 1-f).

Fig. 1. The Characteristics of Pre-webinar Survey Respondents

a: Age distribution of pre-seminar respondents, b: Years of experience as a pharmacist, c: Whether a JASPO member or not, d: Affiliation of respondents, e: Experience of past JASPO seminars, f: Number of attendances at the start-up seminar.

Usefulness and Convenience of the Web-Based Seminar

Respondents’ perspectives on the webinar were recorded using the post-webinar survey. The results, excluding non-responses, are shown in Fig. 2. Most respondents (70.4%) found the difficulty of the e-learning program to be “appropriate,” followed by “slightly easy” (14.2%; Fig. 2-a). Regarding the length of each module (45 min), most respondents found it to be “appropriate” (89.3%), although some (6.4%) considered it to be “slightly long” (6.4%; Fig. 2-b). Majority of the respondents (76.8%) found the e-learning program to be either slightly or significantly improved when compared to the previous seminars held in person (Fig. 2-c). A similar sentiment was observed in the evaluation of the convenience of the e-learning program (Fig. 2-d). Most respondents (76.9%) found that their satisfaction with the e-learning program had slightly or significantly improved, compared with in-person seminars (Fig. 2-e). Furthermore, when asked which seminar format participants would prefer between on-demand delivery or in-person style, 31.2% said they would prefer both, and 64.5% preferred on-demand delivery (Fig. 2-f).

Fig. 2. Effectiveness of Web-Based Educational Program

a: Perceived difficulty of the seminar, b: Perceived time length of seminar, c: Whether this on-demand seminar has been more useful than past in-person style seminars, d: Whether this on-demand seminar has been more convenient than past in-person style seminars, e: Whether this on-demand seminar improved your satisfaction when compared to past in-person style seminars, f: Preference of seminar formats between on-demand and in-person style. g: Overall satisfaction throughout the seminars.

Observational Evaluation of Comprehensive Degree before and after the Webinar

The knowledge scores for each module before and after the webinar are shown in Fig. 3. The median post-webinar knowledge score in all the modules was significantly improved compared to the pre-webinar score (p < 0.001; Fig. 3-a: Cytotoxic anticancer drugs, 3-b: molecular targeted drugs, 3-c: immune checkpoint inhibitors, 3-d: colorectal cancer, 3-e: lung cancer, 3-f: breast cancer, 3-g: palliative care). In particular, the number of respondents with low scores (1–4) decreased drastically for all modules in the post-seminar questionnaire (Fig. 3).

Fig. 3. Box and Violin Plot of the Scores for Each Module before and after the Webinar

a: Cytotoxic anticancer drugs, b: molecular targeted drugs, c: immune checkpoint inhibitors, d: colorectal cancer, e: lung cancer, f: breast cancer, g: palliative care. Wilcoxon signed-rank test **: p < 0.001.

The degree of knowledge of the seven modules was analyzed using PCA (Fig. 4). This method may lead to various self-evaluations; for example, some participants’ evaluations may move from 3 before the seminar to 5 after the seminar, whereas for others it may change from 6 to 8, but both represent the same two-point increase. In order to correct for the relative values, the difference in the score from the pre-seminar questionnaire to the post-seminar questionnaire was analyzed using PCA. The results show that 75.2% of the total information (eigenvalue of 5.27; Fig. 4-a) was condensed in the first principal component (PC1), and the information was presumed to be a general evaluation of the seminar (Fig. 4-b). Then, group A was set as the less-improved group and group B as the improved group (Fig. 4-b), and the age differences (Fig. 4-c), years of experience as a pharmacist (Fig. 4-d), whether or not they are a society member (Fig. 4-e), and experience of past society seminars (Fig. 4-f) was analyzed. Group B consisted of younger age groups, those with less experience as a pharmacist, non-society members, and less experience in past society seminars than group A. These values were significantly different (Pearson’s chi-square test, p < 0.0001).

Fig. 4. Effectiveness of Start-Up Seminar

Two groups were classified by principal component analysis (PCA) using the scores of degrees of knowledge before and after the webinar. Subtracted values from after- to before-scores were analyzed using PCA. a: The result of the eigenvalue of PCA and the percentage of each eigenvalue to the whole. b: PCA results based on seven modules. Improved score group set as B and less-improved group set as A. Comparison of age (c), years of experience as a pharmacist (d), JASPO member or not (e), and experience of past JASPO seminars (f) between groups.

Next, we explored the differences in self-evaluation between the urban and the rural areas. We defined urban prefectures as those with a population of over 500000, and rural prefectures as those with a population of less than that. The urban prefectures included Hokkaido, Miyagi, Saitama, Tokyo, Kanagawa, Niigata, Shizuoka, Aichi, Kyoto, Osaka, Hyogo, Okayama, Hiroshima, Fukuoka, and Kumamoto. There was no difference in self-evaluations between urban and rural prefectures (Table 1).

DISCUSSION

The “start-up seminar” was planned for beginner to moderately skilled pharmacists, and this survey was one of the most extensive questionnaire-based studies on the perspectives and evaluation of an e-learning oncology program for Japanese pharmacists. Although it was planned for young or inexperienced pharmacists, a broad selection of age groups as well as experienced pharmacists participated in the seminar (Fig. 1), as did non-society members. Almost all respondents worked at either general hospitals or pharmacies, and over half of the respondents were first-time participants. Among those that had attended a seminar before, most had done so only once (Figs. 1-e, f). The survey population matched our intentions.

Regarding the usefulness and convenience of the web-based seminar, 70% of respondents found the difficulty level of our e-learning program to be appropriate, and 89.3% of respondents evaluated the time length of 45 min as appropriate. Over 70% of respondents felt that the new format was satisfactory and an improvement when compared to the former in-person seminars (Figs. 2-c–e). Furthermore, over 80% of the respondents felt satisfied throughout the seminars (Fig. 2-g). When asked about the advantages of on-demand seminars, respondents found repeatability, accessibility, and convenience to be strong factors. In fact, registrants from all over the country were able to participate in the seminar despite the coronavirus disease (COVID-19) pandemic.^15,16) Respondents also expressed the expectation that the next seminar would be held either exclusively or supplementarily in the on-demand format. The COVID-19 pandemic may have forced a shift to online seminars, but it also created the opportunity to make these educational seminars more accessible and equal.^15,16) In light of these results, even after the COVID-19 pandemic subsides, we suggest that web-based seminars be conducted alongside traditional in-person events to enable knowledge to be shared more widely.

Figure 3 shows the knowledge scores for each module before and after the webinar. Scores were significantly improved in all modules in terms of the degree of knowledge. It is noteworthy that the number of respondents who had a low score (1–4) decreased drastically after the seminar in all modules (Fig. 3). These results suggest that all of the modules were effective for participants who had low self-evaluations before the seminar.

We used PCA to analyze which groups benefited from the seminar. To standardize the self-assessment, we used the pre-seminar values as a baseline and analyzed the change in the values. PCA results revealed that 75.2% of the total information (eigenvalue of 5.27; Fig. 4-a) was condensed in PC1, and this information was presumed as a total evaluation of the seminar. Moreover, contribution rates of the second, third, and other components were very low (the eigenvalue of PC2 was 0.49 (7.0%), PC3 was 0.4 (5.7%) and the others were less than 5.0%). Consequently, we divided the respondents into two groups based on the results of the PC1, and analyzed which group improved their evaluation after the seminar. Group B consisted of younger age groups, those with less experience as a pharmacist, non-society members, and those with less experience in past society seminars than group A. Results suggest that the “start-up seminar” was meaningful for both beginners and moderately skilled pharmacists, and it served as an effective educational tool to the target audience of the “start-up seminar.”

Furthermore, the analysis of the equalization of self-evaluation between the urban and the rural prefectures city revealed that the web-based seminar provided a uniform learning effect throughout the country.

Overall, our results showed that most respondents were satisfied with the shift to an e-learning program from the former in-person seminar. However, this study has several limitations. First, the questionnaire survey employed for this study evaluated the self-assessment degree of knowledge and not the educational effects by objective measures. Furthermore, it is difficult to assess the elaborate portions that the pharmacists learnt through this method. Though standardized learning achievement-test scores are better than self-assessment test scores to assess the efficacy of the seminar, there are some disadvantages such as decreasing recovery rate, respondent bias (because the seminar attendance was not mandatory), and the fact that the achievement-test cannot be enforced. To evaluate the educational benefits on Japanese pharmacists in charge of oncology, we should develop and apply the most effective assessment score. Second, the recovery rate was high (94.6 and 90.3% in the pre-and post-webinar survey, respectively), but no responses were frequently observed in several questions, especially in the post-webinar survey. The respondents in the post-webinar survey could have enthusiastically answered such a survey after completing all web-based educational program. Therefore, the results of the study might not be generalizable to most pharmacists licensed in Japan.

However, most attendees felt comfortable transitioning from traditional in-person learning experiences to an online learning environment in the current pandemic era. Several studies regarding the effectiveness of online education programs for pharmacists and pharmacy students have also reported similar results.^11,17,18)

We previously reported the effect of webinar for mid- to advanced-level pharmacists¹⁹⁾ and similar results were obtained. However, the field of oncology is diverse, and the content to be learnt becomes deeper according to the number of years of experience. In fact, the percentage of those with less than 10 years of pharmacist experience was over 50% for “start-up” and less than 50% for “brush-up” seminar and number of pharmacists worked in pharmacy was over 30% for “start-up” and less than 30% for “brush-up” seminar. It is meaningful to verify and report the web-based educational effect on each population (start-up and brush-up seminar) in the field of cancer where a wide range of content is learnt.

Therefore, we suggest that the shift to e-learning will be feasible as a method to ensure recurrent education for Japanese pharmacists.

CONCLUSION

The e-learning program was acceptable to Japanese pharmacists as a post-graduate education for pharmaceutical care in oncology. The majority of the participants found e-learning to be satisfactory in acquiring knowledge. Even after the COVID-19 pandemic subsides, education through web-based seminars should be considered valuable.

Acknowledgments

We thank the participants of the “JASPO Start-up seminar” who cooperated in the surveys for this study. We would also like to express our sincere gratitude to Dr. Kato, president of JASPO, and Dr. Yamamoto, Dr. Kondo, and Dr. Matsui, vice presidents of JASPO, for their cooperation in organizing this educational program.

This study was financially supported by the Japanese Society of Pharmaceutical Oncology.

Author Contributions

Hideyuki Terazono: Methodology, formal analysis, writing—original draft preparation.

Masami Tsuchiya, Yosuke Maki: Conceptualization, writing—review & editing.

Naoki Yoshikawa, Yosuke Kawahara, Keiko Nishimura, Keisuke Shinohara, Daisuke Ogawa, Riho Mori, Yoshihiro Iwamoto, Akira Shinohara, Fumio Itagaki, Hiroyuki Masuko, Masahito Yonemura: Writing—review & editing.

Mayako Uchida: Writing—review & editing, supervision.

Conflict of Interest

The authors declare no conflict of interest.

Supplementary Materials

This article contains supplementary materials.

REFERENCES

• 1) Saseen JJ, Ripley TL, Bondi D, Burke JM, Cohen LJ, McBane S, McConnell KJ, Sackey B, Sanoski C, Simonyan A, Taylor J, Vande Griend JP. ACCP Clinical Pharmacist Competencies. Pharmacotherapy, 37, 630–636 (2017).
• 2) Arita E. Why pharmacists need education in the humanities: a medical professional educator’s viewpoint. Yakugaku Zasshi, 140, 411–414 (2020).
• 3) Kelland LR. New platinum antitumor complexes. Crit. Rev. Oncol. Hematol., 15, 191–219 (1993).
• 4) Dilruba S, Kalayda GV. Platinum-based drugs: past, present and future. Cancer Chemother. Pharmacol., 77, 1103–1124 (2016).
• 5) Mauri G, Pizzutilo EG, Amatu A, Bencardino K, Palmeri L, Bonazzina EF, Tosi F, Carlo Stella G, Burrafato G, Scaglione F, Marsoni S, Siravegna G, Bardelli A, Siena S, Sartore-Bianchi A. Retreatment with anti-EGFR monoclonal antibodies in metastatic colorectal cancer: systematic review of different strategies. Cancer Treat. Rev., 73, 41–53 (2019).
• 6) Jacob JB, Jacob MK, Parajuli P. Review of immune checkpoint inhibitors in immuno-oncology. Adv. Pharmacol., 91, 111–139 (2021).
• 7) Fan Y, Geng Y, Shen L, Zhang Z. Advances on immune-related adverse events associated with immune checkpoint inhibitors. Front. Med., 15, 33–42 (2021).
• 8) Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin., 68, 394–424 (2018).
• 9) Ferlay J, Colombet M, Soerjomataram I, Mathers C, Parkin DM, Pineros M, Znaor A, Bray F. Estimating the global cancer incidence and mortality in 2018: GLOBOCAN sources and methods. Int. J. Cancer, 144, 1941–1953 (2019).
• 10) Suzuki S, Uchida M, Suga Y, Sugawara H, Kokubun H, Uesawa Y, Nakagawa T, Takase H. A nationwide survey of community pharmacist contributions to polypharmacy in opioid-using and non-using cancer patients in Japan. Biol. Pharm. Bull., 42, 1164–1171 (2019).
• 11) Patel H, Tan CJ, Toh YL, Danilak M, Alexander M. Educational needs of oncology pharmacists: a survey by the International Society of Oncology Pharmacy Practitioners. J. Oncol. Pharm. Pract., 27, 623–634 (2021).
• 12) Bhatti S, Wake N, Jani Y. Evaluating the effectiveness of digital communication within the national medication safety network for England. Eur. J. Hosp. Pharm., 2021, 1–5 (2021).
• 13) Basheti IA, Nassar R, Barakat M, Alqudah R, Farha RA, Muqatash T, Thiab S, Saini B. Pharmacists’ perceived barriers towards delivering their emergency roles during the COVID-19 pandemic and perceived policymakers’ responsibilities. J. Pharm. Policy Pract., 13, 62 (2020).
• 14) Tsuchiya M, Kawazoe H, Ezura M, Hashiguchi K, Iihara H, Hashimoto H. Evaluation of the outcome of the International Journal Club for Healthcare Professionals and Students. Iryo Yakugaku, 46, 664–671 (2020).
• 15) Gravas S, Ahmad M, Hernandez-Porras A, Furriel F, Alvarez-Maestro M, Kumar A, Lee KS, Azodoh E, Mburugu P, Sanchez-Salas R, Bolton D, Gomez R, Klotz L, Kulkarni S, Tanguay S, Elliott S, de la Rosette J. Impact of COVID-19 on medical education: introducing homo digitalis. World J. Urol., 39, 1997–2003 (2021).
• 16) McMahon CJ, Tretter JT, Faulkner T, Krishna Kumar R, Redington AN, Windram JD. Are e-learning Webinars the future of medical education? An exploratory study of a disruptive innovation in the COVID-19 era. Cardiol. Young, 31, 734–743 (2021).
• 17) Maloney S, Haas R, Keating JL, Molloy E, Jolly B, Sims J, Morgan P, Haines T. Effectiveness of Web-based versus face-to-face delivery of education in prescription of falls-prevention exercise to health professionals: randomized trial. J. Med. Internet Res., 13, e116 (2011).
• 18) Singhi EK, Dupuis MM, Ross JA, Rieber AG, Bhadkamkar NA. Medical hematology/oncology fellows’ perceptions of online medical education during the COVID-19 pandemic. J. Cancer Educ., 35, 1034–1040 (2020).
• 19) Tsuchiya M, Terazono H, Maki Y, Yoshikawa N, Kawahara Y, Nishimura K, Shinohara K, Ogawa D, Mori R, Iwamoto Y, Itagaki F, Masuko H, Yonemura M, Uchida M. Evaluation of a web-based educational programme for pharmacists during the COVID-19 pandemic in Japan. J. Clin. Pharm. Ther., 46, 1743–1749 (2021).