2020 Volume 4 Article ID: 2020-039
基礎学力向上を目的とした有機化学の予習動画を用いた講義型補講と問題演習型補講を実践し,各補講の教育効果を評価した.補講前の成績により学生を四分位で成績上位から第1~4群にわけた.その各群を,さらに補講受講者と非受講者に分類し,補講後の成績を比較した.予習動画を用いた講義型補講では,第3群で受講者の成績が非受講者より良好であった.また,予習動画を用いた講義型補講について実施したアンケートより,停止,巻戻し機能等の動画の特性を生かした語句があったことから,学生の理解度に応じた学習が個々に行われたことが考えられた.問題演習型補講では第2群で受講者の成績が非受講者より不良であったことから,この群では補講を受講せずに自己学習をした方が有効であったことが示された.本研究では,補講の学習方略や補講受講前の成績によって,得られる学習効果は異なることが明らかとなった.
This study evaluated the learning effects of two interventions aimed at improving the students’ basic academic skills, namely by lecture-type supplementary classes that used preparation videos on organic chemistry and supplementary classes delivered in the drill-and-check format. Students were divided into four quartile groups in the order of their pre-intervention grades. Each group was further divided into an intervention and a non-intervention subgroup to compare the two groups’ post-intervention examination results. In the lecture-type supplementary classes that used preparation videos, Group Three’s scores for the intervention subgroup were superior to those of the non-intervention subgroup. Furthermore, the questionnaire responses on the video-based supplementary classes revealed keywords that indicated some video-specific characteristics, such as the pause and rewind functions, suggesting that the independent learning style matched the level of understanding. In the drill-and-check-type supplementary classes, Group Two’s scores for the intervention subgroup were inferior to those of the non-intervention subgroup, indicating that, for this group, self-study was more effective than supplementary classes. The study showed that the learning effects obtained vary depending on the supplementary class strategy and the students’ grades before the intervention.
Due to a declining birthrate and an upward trend in enrollment rates, Japan is approaching an era of full enrollment in higher education. Against this backdrop, only about 80% of students entering university graduate within their term of study, while 10% withdraw before graduation. With a decreased sense of purpose and motivation to learn, the environment is becoming increasingly challenging for universities1). Faculties of pharmacy are no exception, and some pharmaceutical universities have identified the increase in dropout and repeater rates as issues that need to be addressed2). Meanwhile, pharmacists are expected to acquire a high degree of expertise and fulfill a variety of roles in society as leaders of medicine and scientists of the streets3), and, to accomplish these roles, it is essential that they develop basic academic skills through high-quality pharmaceutical education. To enhance the motivation to learn and encourage students who lack a sense of purpose, it is crucial to consider introducing programs that, from the first year of study, help students imagine their work as a pharmacist in the future and foster willingness to learn independently1).
As a way of correcting gaps in basic academic skills that arise despite such efforts, it is considered effective to provide supplementary education that can be undertaken flexibly in accordance with students’ academic abilities and characteristics. However, initiatives to provide supplementary classes that are unsystematic and fail to consider its effects on learning have the potential to exhaust students and teachers; therefore, an approach that seeks to consider and provide supplementary classes that are optimized to the needs of the students are necessary for students and teachers who are asked to accomplish much. Furthermore, to improve academic ability and promote active learning among students, it is considered useful to evaluate the educational effects of such interventions and apply the findings in a way that benefits students.
In pharmaceutical education, there have been some reports on the learning effects of supplementary classes4,5) and classes that use preparation videos6); however, there has been insufficient research on supplementary classes for students in the upper years of university. The authors have previously evaluated the learning effects of drill-and-check-type supplementary classes for fourth- and fifth-year students implemented in academic year (AY) 2015 and 2016 using the results of mock examinations performed before and after an intervention7). This present study used the results of regular examinations before and after the intervention to analyze the learning effects of two different forms of supplementary classes aimed at improving basic academic skills for organic chemistry provided to fourth-year pharmacy students in AY 2018. The purpose of the study is to contribute to the optimization of supplementary classes by evaluating what kind of learning effects are observed among different groups of students.
In September and October 2018, lecture-type supplementary classes that incorporated preparation videos on the topic of organic chemistry (hereinafter “video-based supplementary lectures”) were provided to fourth-year students, who wished to participate, in the Faculty of Pharmacy at Osaka Ohtani University. Additionally, since in the previous year the attendance rate for lecture-type classes that did not use preparation videos was low among students with lower grades, 30 students with the lowest grades in physics and chemistry from year one to three were required to attend the classes. An overview of the video-based supplementary lectures is provided in Table 1, along with details of the evaluation method. Two 90-minute lectures were held per day, one to two times per week, for a total of five days, and students were encouraged to prepare for the lectures on days two to five by viewing videos. The videos introduced basic content related to each lecture theme (stereochemistry, nucleophilic addiction/substitution, acid-base strength, etc.), and four 10-minute videos were available for viewing through Moodle (3.5.5), the university’s learning management system. During the classes, students received instruction on more advanced content for each theme, performed drill-and-check exercises, and received explanatory lectures.
Video-based supplementary lectures | Drill-and-check-type supplementary classes | |
---|---|---|
Examinations used to form the quartile groups | Total score for three organic chemistry subjects taken in the second and third year | |
Intervention period | September to October 2018 | December 2018 |
Learning strategy | Day 1 (two 90-minutes sessions/day) | Sessions 1 to 5 |
Orientation Lecture, drill-and-check exercise, explanatory lecture |
Drill-and-check-type exercises using compulsory questions from the national pharmacist examination | |
Sessions 2 to 5 (two 90-minutes sessions/day) | Session 6 | |
[Before class] View videos on basic content [During class] Lecture on more advanced content, drill-and-check exercise, explanatory lecture |
Drill-check-exercise for questions with a low rate of correct answers identified from the results of sessions 1 to 5 | |
Evaluation method (Jan. 2019) | Score in examination (organic chemistry) Questionnaire |
Score in examination (organic chemistry) |
Students were assigned to one of the four quartile groups based on their total score for examinations in Organic Chemistry courses (Basic Chemistry, Organic Chemistry I, and Organic Chemistry II), which are compulsory in the second and third years. Each group was further divided into an intervention subgroup, comprising students who took the video-based supplementary lectures, and a non-intervention subgroup, and the intervention was evaluated based on students’ result for the organic chemistry section of the examination (January 2019). Table 2 shows the number of participants in the intervention and non-intervention subgroups of the quartile groups for each supplementary class type.
Group 1 | Group 2 | Group 3 | Group 4 | Total | ||
---|---|---|---|---|---|---|
Video-based supplementary lectures | Intervention subgroup | 18 | 17 | 23 | 28 | 86 |
Non-intervention subgroup | 20 | 21 | 11 | 9 | 61 | |
Drill-and-check-type supplementary classes | Intervention subgroup | 10 | 12 | 8 | 6 | 36 |
Non-intervention subgroup | 28 | 26 | 26 | 31 | 111 | |
Students who took both types of supplementary classes | 7 | 10 | 8 | 3 | 28 | |
Students who took only video-based supplementary lectures | 10 | 7 | 15 | 25 | 57 | |
Students who took only drill-and-check-type supplementary classes | 3 | 2 | 0 | 3 | 8 | |
Students who no took both types of supplementary classes | 18 | 19 | 11 | 6 | 54 |
After completing the fifth day of lectures, a questionnaire on the video-based supplementary lectures was administered (Table 3). The questionnaire comprised 10 questions; students who reported having viewed two to four of the preparation videos in Q1 were instructed to answer Q2 to Q5 and Q7 to Q10; students who had watched one or none of the videos were instructed to answer Q6 to Q10. Q9 asked students to state whether the videos or video-based lectures should be improved, and Q10 was an open-ended item with space for students to write any other opinions or impressions they had. Co-occurrence network analysis was performed to understand connectivity between keywords extracted from these free responses.
Question item | Group 1 | Group 2 | Group 3 | Group 4 | Total |
---|---|---|---|---|---|
Q1: No. of videos viewed | |||||
4 | 3 | 4 | 7 | 5 | 19 |
3 | 6 | 2 | 7 | 5 | 20 |
2 | 4 | 1 | 2 | 4 | 11 |
1 | 2 | 3 | 2 | 4 | 11 |
0 | 3 | 3 | 3 | 6 | 15 |
Q2: Level of understanding of the videos | |||||
I understood. | 3 | 3 | 1 | 0 | 7 |
I understood to some extent. | 8 | 3 | 10 | 8 | 29 |
I cannot say. | 2 | 0 | 4 | 2 | 8 |
I understood hardly anything. | 0 | 1 | 1 | 2 | 4 |
I did not understand. | 0 | 0 | 0 | 2 | 2 |
Q3: Impression of the length of each video (approx. 10 minutes) | |||||
I would prefer them to be longer. | 0 | 1 | 0 | 2 | 3 |
I would prefer them to be a little bit longer. | 2 | 1 | 2 | 3 | 8 |
They were suitable. | 10 | 4 | 14 | 9 | 37 |
I would prefer them to be a little bit shorter. | 1 | 1 | 0 | 0 | 2 |
I would prefer them to be shorter. | 0 | 0 | 0 | 0 | 0 |
Q4: Device used to view the videos | |||||
Computer at home. | 1 | 1 | 0 | 1 | 3 |
Smartphone/tablet. | 12 | 6 | 14 | 13 | 45 |
Computer at school. | 0 | 0 | 2 | 0 | 2 |
Q5: Good points about the video-based classes (Multiple responses allowed): Only those who answered that the number of videos viewed in Q1 is 1 or 2 | |||||
It was easier to study at my own pace than lectures. | 6 | 5 | 12 | 13 | 36 |
I spent more time studying outside of class. | 6 | 0 | 7 | 4 | 17 |
It helped me form the habit of studying outside of class. | 1 | 2 | 2 | 2 | 7 |
It was good to spend a long time on drill-and-check exercises in class. | 3 | 1 | 2 | 1 | 7 |
I can view the videos multiple times, which I think will help me improve my grades. | 11 | 6 | 11 | 12 | 38 |
Class time was used efficiently. | 13 | 5 | 11 | 3 | 31 |
I did not see any advantages. | 0 | 0 | 0 | 0 | 0 |
Q6: Reasons for not viewing the videos (multiple responses allowed) | |||||
I did not have time to view the videos. | 2 | 1 | 0 | 6 | 9 |
It was troublesome. | 0 | 0 | 0 | 0 | 0 |
The videos were difficult to watch. | 0 | 0 | 0 | 1 | 1 |
I was dissatisfied with the content of the videos. | 0 | 0 | 0 | 0 | 0 |
There was no environment (device, etc.) nearby in which I could view the videos. | 1 | 0 | 1 | 3 | 5 |
Other. | 2 | 2 | 2 | 2 | 8 |
I did not see any advantages. | 0 | 0 | 0 | 0 | 0 |
No answer | 0 | 3 | 2 | 2 | 7 |
Q7: Whether students wanted to watch other videos on organic chemistry | |||||
Yes. | 10 | 8 | 12 | 15 | 45 |
Yes, depending on the content. | 7 | 1 | 5 | 3 | 16 |
Either would be fine. | 1 | 3 | 1 | 2 | 7 |
I do not really want to watch them. | 0 | 0 | 0 | 0 | 0 |
I do not want to watch them. | 0 | 0 | 0 | 0 | 0 |
No answer | 0 | 1 | 3 | 4 | 8 |
Q8: About video-based classes in the future | |||||
I would like them to be incorporated. | 10 | 7 | 7 | 13 | 37 |
I would like them to be incorporated partially. | 3 | 3 | 11 | 6 | 23 |
Either would be fine. | 5 | 2 | 1 | 1 | 9 |
I do not really want them to be incorporated. | 0 | 0 | 0 | 0 | 0 |
I do not want them to be incorporated. | 0 | 0 | 0 | 0 | 0 |
No answer | 0 | 1 | 2 | 4 | 7 |
Q9: About the videos or video-based lectures should be improved | |||||
Q10: About other opinions or suggestions |
Responses to “others” in Q6: “I did not know”, 4 students; “When I tried to rewind, it went back to the beginning, so I gave up”, 1 student; “I forgot”, 1 student; no response, 2 students.
In December 2018, drill-and-check-type supplementary classes were provided to fourth-year students, who wished to participate in the study, in the Faculty of Pharmacy at Osaka Ohtani University. An overview of the drill-and-check-type supplementary classes in AY 2018 is provided in Table 1, along with details of the evaluation method. A total of six 90-minute drill-and-check classes were conducted at a pace of two classes per week. In each of the first five sessions, students tackled 90 questions that appeared as compulsory questions in national examinations over the past five years (total of 450 questions); in the final session, the 90 questions with the lowest rate of correct answers were identified and re-assigned. At the end of each session, an explanatory booklet was provided to students, and scores were shown to students who requested to see them.
In addition to the four groups formed in Strategy 1, students were further divided into an intervention subgroup, comprising students who attended the drill-and-check-type classes, and a non-intervention subgroup. Similar to Strategy 1, results for the organic chemistry section of the examination were used to evaluate the learning effects. It should also be noted that although the results of a mock examination were used to evaluate students before and after interventions in AY 2015 and 2016, which used similar drill-and-check classes, in the present study a different examination was used in the evaluation. Compared with the knowledge-recall-type mock examinations used in AY 2015 and 2016, the examinations used in the present study—examinations on organic chemistry conducted in the second and third year of study and the fourth-year examination (organic chemistry)—contained more problems that required students to think; for example, there were several problems that could not be solved without an understanding of chemical reaction mechanisms. In the evaluations that used the results of mock examinations, the scores used in the evaluation also included other subjects; however, to facilitate evaluation of basic academic skills for organic chemistry, this time the examinations were changed to organic chemistry in the second and third years of study and the examination (organic chemistry) in the fourth year.
3. Statistical analysis and ethical considerationsJMP (14.2.0) was used for statistical analysis. In this study, a p value of less than 0.05 was considered statistically significant. It was explained to students that the questionnaire results would be used in the study, and that they would not suffer any disadvantage if they declined to participate. Students were given opportunities to decline participation. The study was approved by the Bioethics Committee of the Faculty of Pharmacy at Osaka Ohtani University (BE-0051-20).
For video-based supplementary lectures, examination scores (organic chemistry) for the intervention and non-intervention subgroups are shown in Figure 1A. In group three, examination scores (organic chemistry) were significantly higher for the intervention subgroup than for the non-intervention subgroup. Conversely, in groups one, two, and four, no significant differences were observed between the intervention subgroup and non-intervention subgroup in the examination (organic chemistry).
Scores before the intervention (total score for examinations in Organic Chemistry in the second and third years) and after the intervention (score for the Organic Chemistry section of the examination in January 2019) for the non-intervention subgroup (–) and the intervention subgroup (+) of each quartile group. A: Video-based supplementary lectures. B: Drill-and-check-type supplementary classes.
Questionnaire responses were obtained from 76 of the 85 students who undertook the video-based supplementary lectures, giving a response rate of 91%. The questionnaire frequency results for each quartile group are shown in Table 3. For Q1, 19 students (approx. 25%) viewed all of the videos; 26 students (approx. 34%) viewed one or none of the videos, with group four accounting for a higher proportion than other groups at 38%. For Q3, 37 students (74%) stated that the length of each video (approx. 10 minutes) was “appropriate”; for Q4, about 90% of students stated that they used a smartphone or tablet to view the videos. Nine (approx. 45%) of the students who gave reasons (20 responses) for not watching the videos in Q6 stated, “I did not have time to view them,” with group four accounting for a higher proportion than other groups at 66%.
The free-response questions (Q9 and Q10) are listed in Tables 4 and 5. For Q9 (23 responses), words related to difficulties in reading the text in the videos were identified. For Q10 (19 responses), the advantage of utilizing the characteristics of video became clear, such as pause and rewind functions.
Category | Q9 (How can the videos or video-based lectures be improved. Please write the details here.): |
---|---|
Difficulty reading text and hearing voices on video | I couldn’t adjust the volume when I saw it on my school PC, so I felt it was inconvenient when I wanted to make a slightly louder sound. (Group 1) I thought I should make the volume a little louder. (Group 1) It was quite shaken, so I want you to eliminate it. (Group 1) Sometimes it was difficult to see the figures and formulas that were written, so I would like you to take a close-up. (Group 1) I want you to write a little bigger. (Group 3) I often watch it on my smartphone, so I want you to be able to watch the video on the full screen. (Group 3) It was difficult to understand the difference between the black pen and the blue or green pen. (Group 3) The letters were a little small and hard to see. (Group 3) With a ballpoint pen, red and blue were thin and hard to see, so a thicker pen is better. (Group 3) I thought it would be nice to use it when I want to remember the basics and preparations before taking classes. I thought that the most important part could be expressed in letters. (Group 3) I want you to make the letters a little bigger and make the pen thicker. (Group 3) I would appreciate it if you could enlarge the part written by the teacher and copy it. (Group 4) I want you to project larger in a brighter environment. If possible, use a whiteboard. (Group 4) I would appreciate it if you could raise the volume of your voice. (Group 4) It was inconvenient to compare it with my own print because there were times when the edges of the print were not visible. (Group 4) After all, the whiteboard was better than the above. It was good depending on the color of the pen. (Group 4) I want you to be able to enlarge it. Sometimes the characters are hard to see. (Group 4) |
Video content | The summary should be at the end. (Group 1) I sometimes feel that the tone is a little quick in the video, but I think it can’t be helped because of the video time. (Group 3) |
Other | I want to hear the range that I have never done at the center. (Group 1) I’ll forget to see it, so I hope you’ll tell me like a notification every time. (Group 2) I didn’t know how much to write on the print, so I didn’t put it clean, but everything else was good. (Group 2) I would be happy if the data could be saved in the mobile phone. (Group 2) I will watch the video firmly from the next. (Group 3) |
Category | Q10 (If you have any other opinions or suggestions, please write them here): |
---|---|
Effective use of time | I was grateful to be able to watch them easily for a short time. (Group 1) I thought it was amazing how it was so easy to understand in just 10 minutes. (Group 1) They were about the right length to watch on the train, so that was good. I could watch them on the train in the morning, so I could use my time effectively. (Group 1) It was good because I could also learn just before going to sleep. (Group 1) I was able to prepare before class, so the classes were easy to understand. They were really easy to listen to, and the explanations were easy to understand, so I learned a lot. (Group 2) It was great that I could watch them during my commute and review them any number of times. (Group 4) |
Video characteristics (stop, repeat, etc.) | I forget organic really quickly, so it was really helpful to have videos so I could go back and see. (Group 1) It was very good because I often forget things even if I understand them in class, and I could go back and confirm things I was not sure about. (Group 1) They were really easy to use because you can stop and think if it gets complicated, unlike in class. (Group 2) Because it was a video, I was able to stop and rewind it myself, and it was great to be able to go back and watch them again. (Group 3) The best thing was that I could do it at my own pace. (Group 3) I was glad because I could look at the bits where I struggle at my own pace during pre-study before the class, and that helped me to learn. (Group 3) I could go back and watch the parts I missed any number of times, so I understood it properly as I went through, which helped me to master the content. (Group 4) |
Video content | The videos were also really easy to understand. Thank you. (Group 2) I was glad to be able to check the basic information. (Group 3) The length was just right. (Group 3) |
Other | If you upload other videos, I will save them all. I think it is hard to create videos when you are busy, so please do your best. (Group 1) I also want to see explanations of difficult questions, like applied questions. I want to see videos for other subjects, too. (Group 2) I want to know how to solve and think about problems that involve different fields. (Group 4) |
For drill-and-check-type supplementary classes, the examination scores (organic chemistry) for the intervention and non-intervention subgroups are shown in Figure 1B. In group two, the examination scores (organic chemistry) were significantly lower for the intervention subgroup than for the non-intervention subgroup. Furthermore, in groups one, three, and four, no significant differences in scores were observed between the intervention and non-intervention groups.
In group one (Fig. 1), no significant differences were observed between the scores of students who took the supplementary classes and those who did not for either the video-based supplementary lectures or the drill-and-check-type supplementary classes. This suggests that the academic ability of students in group one is not low enough to necessitate provision of care through the two types of supplementary classes.
In group two, video-based supplementary lectures did not affect learning because, as shown in Figures 1A, there were no significant differences between the scores of students who took the video classes and those who did not. Moreover, for drill-and-check-type supplementary classes, students in group two who took the classes had significantly lower scores than those who did not. This result is opposite to the results obtained under a similar strategy in AY 2015 and 2016, where scores on knowledge-recall-type mock examinations were used in the evaluation, in which students who took supplementary classes achieved superior grades to those who did not7). The compulsory questions on the national pharmacist examination that were used in the drill-and-check-type supplementary classes include many knowledge-recall-type questions. It would appear that in AY 2015 and 2016, students who took the supplementary classes saw their scores improve on the mock examinations used in the evaluation, which contains a large number of knowledge-recall-type questions, as increased opportunities for output helped students establish knowledge. Conversely, compared with the mock examinations, more questions on the examination (organic chemistry) used in the evaluation of the present study required students to think through problems to solve them; therefore, learning effects were limited among students who took the drill-and-check-type supplementary classes. Their scores tended to be inferior to those of students who did not take the classes but studied independently. As a means of enhancing the learning effects of drill-and-check-type supplementary classes, the timing of the intervention could be adjusted, for example, by providing supplementary classes after students have better understood the approach of organic chemistry.
In groups three and four (Fig. 1B), drill-and-check-type supplementary classes did not affect learning because there were no significant differences between the scores of students who took the supplementary classes and those who did not. However, in group three, for video-based supplementary lectures, the scores of those who took supplementary classes were superior to those who did not (Fig. 1A). In our previous work, no difference in the scores of the group three was observed between students who took drill-and-check-type supplementary classes, which we attributed to a lack of understanding of the concept of organic chemistry7). The reason for the difference in the results of this study is that the students were able to learn the concepts of organic chemistry by utilizing the characteristics of the video clips (e.g., stopping and rewinding functions), they were able to understand the concepts of organic chemistry better. However, in groups one and two, there was no difference between the supplementary lecture group and the non-student group. This is because of the understanding of the concept of organic chemistry without lecture-type classes that use preparation videos. It was also considered that this was a group that had already been deepened. It was suggested that video-based supplementary lectures such as those conducted in this study did not have a positive effect on the grades of all students.
In group four, even for video-based supplementary lectures, no significant differences were observed between the scores of students who took the supplementary lectures and those who did not. One reason for this could be that students were unable to achieve effective self-study when they studied independently. Given that more students reported that they had watched one or none of the videos than other groups (Q1), and more students stated, “I did not have time to view them” as a reason for not viewing the videos (Q6), it would seem necessary to improve the way in which students approach their learning.
Furthermore, Mohammad Qayoumi of Stanford University highlights the need for peer pressure8) to deal with students who do not prepare before coming to class, which suggests that it is necessary to find ways to compel students to view videos through, for example, the use of in-class activities that require students to work in groups. Thus, for group four, we will improve the strategy used and consider ways of improving the effectiveness of the teaching methods. It should also be noted that almost all of the 30 students who were required to take the supplementary classes were assigned to group four, and although the attendance rate improved compared with the same group in the previous academic year (data not shown), no significant differences in results were observed when compared with non-intervention groups.
For Q9 of the questionnaire responses, many students mentioned that the videos were difficult to watch (Table 4), about 90% of students used a smartphone or tablet to view the videos (Table 3). Therefore, we will aim to improve the videos to ensure easy viewing on a small screens.
The study has shown that the learning effects obtained vary depending on the strategy of the supplementary classes and students’ grades prior to intervention. Video-based supplementary lectures facilitated learning that corresponded to each student’s level of understanding, which was difficult to achieve through traditional lectures and drill-and-check-type classes due to time and personnel constraints; therefore, the intervention can be considered significant for those students whose learning was not always effective. However, video-based supplementary lectures require time for preparation, and introducing them in an unsystematic manner has the potential to encroach on self-study time set aside for other activities, such as reviewing. Because the learning effects of each type of supplementary class vary depending on students’ grades prior to intervention, it would be effective for teachers to demonstrate to students—with recourse to evidence—that, in view of their grades, there may be supplementary classes they need to take and to encourage students to take supplementary classes that correspond to their needs. Moreover, drill-and-check-type classes for knowledge-recall-type questions were found to have limited learning effects for questions that require students to think through problems. This suggests that effective learning strategies and timing also differ depending on the purpose of the learning, which could be to promote understanding through thinking or to promote the consolidation of knowledge. For supplementary classes to meet the needs of students and effectively promote learning, it is important that they are matched to student characteristics, such as academic ability; therefore, in addition to implementing effective supplementary classes at specific academic levels, it would be effective to consider the use of more effective learning strategies and increase the range of options available. Thus, through education tailored to students’ needs, it should be possible to eliminate gaps in basic academic skills among students. Going forward, we will investigate more cases, continue evaluating the learning effects of supplementary lectures designed to enhance basic academic skills, and consider more effective learning strategies with the goal of developing optimal forms of supplementary classes for students.
There are no conflicts of interest to disclose.
We would like to thank everyone in the Laboratory of Pharmaceutical Education of Setsunan University, who provided valuable advice regarding the statistical analysis method used in this study.