Relationship between examination score and homework completion time for on-demand classes among pharmacy students during COVID-19

抄録

COVID-19流行下では対面講義からオンライン教育への移行が見られた．本研究では2020年にオンデマンド講義として開講した薬物治療学IIを対象に，学生の課題取り組み時刻と当該科目の試験成績との関係性について検討し，学生の時間管理についての評価を行った．対象は139名の薬学3年生で，Google Formsを介して出された課題に対し，学生の課題送信履歴からその課題に取り組んだ期日と時刻を抽出した．その上で試験成績と課題に取り組んだ期日や時刻との関係性を検討した．その結果，締め切り日に課題を提出した学生の課題提出時刻は遅く，試験成績も低かった．また課題を初日に提出した学生の中で，科目開講時刻から近い時間帯に課題を提出した学生の試験成績は高かったが，開講時刻から離れた遅い時刻に課題を提出した学生の試験成績は低かった．これらの結果から，一部の学生はオンデマンド講義に対する時間管理に問題があり，学習理解度に影響を与えている可能性が考えられた．

Abstract

Objective: During the Covid-19 pandemic, there was a shift from face-to-face to online lectures. This study investigated the relationship between examination scores and the submissions of lecture-related homework assignments to evaluate student time management. Methods: The 139 participants were third-year pharmacy students who received seven Pharmacotherapeutics II lectures on-demand from September to November 2020. They were classified into three groups based on the time of assignment submission as a submission on the day of lecture (“early” n = 42), up to the day before the deadline (“intermediate” n = 58), and on the deadline day (“late” n = 39). Spearman’s rank correlation coefficients and multivariate regression analysis were used to determine the influence of homework submission times on examination scores. Results: The analysis showed that examination scores were negatively impacted by late submission. Students who submitted their assignments soon after the lectures had higher exam scores, while those who submitted them later had lower scores. Conclusion: The results suggested that some pharmacy students may have poor time management and a tendency toward academic procrastination with on-demand classes, directly affecting their exam scores. Therefore, faculty must communicate with online students about time management in submitting homework assignments soon after lectures and monitor student progress throughout the course.

Introduction

Until 2019, almost all universities in Japan conventionally taught using face-to-face classes. However, COVID-19 has led to major changes in how university lectures are delivered. Since the beginning of COVID-19, most university lectures have shifted from face-to-face classes to remote education¹⁾. In the Faculty of Pharmaceutical Sciences at Sojo University, on-demand education started in May 2020; no lectures were delivered in April 2020. On-demand education is a common remote education method, whereby lecture materials or videos are uploaded onto a platform for students to use in their studies. In this way, university students gain a certain degree of freedom regarding when and how they wish to learn. However, it is possible that remote education deepens the tendency of academic procrastination among students²⁾. Such procrastination involves students delaying academic duties and responsibilities or avoiding them until the last minute³⁾. On-demand education does not set designated times for students to complete tasks; thus, academic procrastination could become a serious problem. To reduce such negative tendencies, it would be expedient for students to plan appropriately; to do so, however, requires self-efficacy⁴⁾. Self-efficacy is a person’s capacity to execute a plan for producing performance attainments⁴⁾; it has been observed that academic procrastination and self-efficacy are negatively correlated⁵⁾. Therefore, it is necessary to find a way to support or enhance students’ self-efficacy. Homework can play an important role in developing self-efficacy; homework reportedly has a positive relationship with self-efficacy⁶⁾. In this regard, many students typically tend to procrastinate with their homework⁷⁾. It has been found that both homework time and time management related to homework are associated with academic achievement such as good examination results⁸⁾.

The implications of the above findings are that students may not take on-demand education seriously: students become concerned about submitting homework but not about properly understanding lecture content. This tendency indicates poor time management and academic procrastination. To verify the existence of these tendencies, it is necessary to ascertain the time information that university students spend on homework with on-demand education. However, previous research, including the studies cited above, has mainly taken the form of questionnaire surveys among students; thus, the results have depended on the accuracy of students’ self-assessments. Accordingly, in this study, by using the transmission history for homework assignments with Google Forms, we investigated the time of day pharmacy students worked on and submitted those assignments. Our focus was the relationship among time of day for completing lecture-related homework assignments and end-of-semester examination score.

Method

1.  Subject selection and eligibility criteria

The study participants were 139 third-year university students (male, 57; female, 82) in the Faculty of Pharmaceutical Sciences at Sojo University, Japan. We selected Pharmacotherapeutics II course for this study; we did not set exclusion criteria.

2.  Lecture and homework content

We created lecture materials in PDF format, which we converted into Microsoft PowerPoint slides. We wrote down all content that would have required explanation in face-to-face classes, and we incorporated that text among the lecture materials in PDF format. These measures were performed according to the guideline for COVID-19 of the Ministry of Education, Culture, Sports, Science and Technology and was accepted at Sojo University. We uploaded the PDF files for each lecture to a learning management system (WebClass), which was developed for Japanese universities (Data Pacific Japan, Ltd, Tokyo, Japan), and the PDF lecture files were uploaded at 07:00 each Tuesday. The university credit for the Pharmacotherapeutics II course was 1, and it comprised seven lectures. Each lecture had two homework assignments (assignments 1 and 2); each assignment comprised five multiple-choice questions (MCQs) (i.e., 10 questions per lecture). With each MCQ, students had to select one from among five answer items. The content of the two assignments for each lecture corresponded to that of the lecture itself. Accordingly, students should have been able to choose the correct answer. If a student answered the MCQs for all the assignments correctly, the total homework assignment score was 70 points. These homework assignments were generated using Google Forms. Students had to submit answers for the two homework assignments within 6 days (Tuesday to Sunday) from the uploading of each lecture; an exception was the seventh lecture, which could be completed within 13 days. After all the lectures had been delivered, the end-of-semester examination for the Pharmacotherapeutics II course was held on November 17, 2020. The examination mainly comprised advanced questions (in MCQ and description formats) based on the assignments. The maximum examination score was 100 points, and we used that score for the course grade.

3.  Statistical analysis

For each lecture, we collected information on the time of day when the students submitted their homework assignments, the number of hours from uploading the lecture to each homework assignment submission time, and the time lag (minutes) between assignments 1 and 2; we did so by employing the transmission history of the homework assignments with Google Forms. For each student, we obtained the above three time-related variables for each lecture. We took the median for all seven lectures as the representative value and assumed that the variables were continuous. In our analyses, we categorized the students into three subgroups based on the number of hours from uploading a lecture to when assignment 2 was submitted. After lecture upload on Tuesday, students who submitted assignment 2 on that day were classed as the “early” group; those who did so from Wednesday to Saturday constituted the “intermediate” group; and those who did so on the deadline day (Sunday) were the “late” group. For continuous variables, we conducted the Kruskal-Wallis test and employed the pairwise Wilcoxon rank sum test with Bonferroni correction. For categorical variables, pairwise Fisher’s exact test with Bonferroni correction was performed. The analysis results appear in Table 1. The time lag between assignments 1 and 2 was log-transformed to approximate a normal distribution. We examined Spearman’s rank correlation coefficients among examination score, total homework assignment score, grade point average (GPA), and submitted time of day of assignment (Table 2A). GPA was calculated based on the course grades for all required subjects in the first 2 years of university. To investigate the relationship between examination score and submitted time of day of assignment excluding the influence of total homework assignment score and GPA, we performed multivariable regression analysis (Table 2B). We assumed each of those analyses to be independent. There were no missing values in our analyses. All analyses were performed using R version 4.0.2 with statistical significance set at P < 0.05.

Table 1 Characteristics of the pharmacy students in total and in the three groups

	Total	Early	Intermediate	Late	P-value
	n = 139	n = 42	n = 58	n = 39	Early vs Intermediate	Early vs Late	Intermediate vs Late
Age (years), median	21.0 (20.0, 21.0)	21.0 (20.0, 21.0)	21.0 (20.0, 21.0)	21.0 (20.0, 21.0)	0.57	1.00	1.00
Females/males, n	82/57	25/17	30/28	27/12	1.00	1.00	0.29
Time lag between assignment 1 and 2 (minutes), median (IQR)	14.0 (6.5, 23.0)	14.8 (9.1, 21.0)	15.0 (6.0, 24.8)	12.0 (4.5, 19.5)	1.00	0.79	0.70
Total homework assignment score, median (IQR)	45.0 (30.0, 58.5)	53.5 (35.3, 60.5)	45.5 (30.0, 58.0)	39.0 (28.0, 54.5)	0.68	0.22	1.00
Examination score, median (IQR)	70.0 (57.5, 82.0)	70.5 (60.0, 83.0)	76.0 (66.3, 82.0)	57.0 (49.0, 73.0)	1.00	< 0.001	< 0.001
GPA, median (IQR)	2.38 (1.88, 2.92)	2.75 (2.28, 3.17)	2.59 (2.02, 2.92)	1.98 (1.69, 2.49)	0.30	0.001	0.011
Assignment 1
Submitted time (hour) of day, median (IQR)	16:30 (10:36, 19:39)	9:59 (9:25, 16:26)	16:50 (12:36, 19:03)	19:34 (15:59, 22:00)	0.003	< 0.001	0.002
Assignment 2
Submitted time (hour) of day, median (IQR)	16:28 (10:26, 19:56)	10:26 (9:43, 17:25)	16:22 (12:07, 19:17)	19:29 (16:06, 22:07)	0.15	< 0.001	0.001

Abbreviations: IQR = interquartile range. GPA = grade point average. Early group indicates students who submitted their assignments on the day of lecture upload. Intermediate group indicates students who submitted their assignments between the 2nd day and the day before the deadline. Late group indicates students who submitted their assignments on the deadline day.

Table 2A Correlation matrix of Spearman’s rank correlation coefficient between bivariate values

	Examination score	Total homework assignment score	GPA	Submitted time of day of assignment
Early group
Examination score
Total homework assignment score	0.68 (< 0.001)
GPA	0.62 (< 0.001)	0.75 (< 0.001)
Submitted time of day of assignment	–0.59 (< 0.001)	–0.37 (0.014)	–0.27 (0.08)
Intermediate group
Examination score
Total homework assignment score	0.27 (0.038)
GPA	0.62 (< 0.001)	0.52 (< 0.001)
Submitted time of day of assignment	0.03 (0.84)	–0.06 (0.64)	–0.04 (0.74)
Late group
Examination score
Total homework assignment score	0.24 (0.14)
GPA	0.68 (< 0.001)	0.52 (< 0.001)
Submitted time of day of assignment	–0.22 (0.18)	–0.29 (0.08)	–0.26 (0.11)

Table 2B Results of multivariable regression analysis

	Objective variable	Explanatory variables	Estimate	P-value	95% CI
Early group	Examination score	Total homework assignment score	0.33	0.043	(0.02, 0.63)
		GPA	7.63	0.037	(0.72, 14.53)
		Submitted time of day of assignment	–0.02	0.001	(–0.04, –0.01)
Intermediate group	Examination score	Total homework assignment score	–0.05	0.69	(–0.29, 0.19)
		GPA	15.18	< 0.001	(9.30, 21.06)
		Submitted time of day of assignment	0.00	0.44	(0.00, 0.01)
Late group	Examination score	Total homework assignment score	–0.09	0.58	(–0.41, 0.23)
		GPA	18.94	< 0.001	(10.20, 27.69)
		Submitted time of day of assignment	–0.01	0.19	(–0.03, 0.01)

Abbreviations: GPA = grade point average, CI = confidence interval. Submitted time of day of assignment indicates the submission time for assignment 2. Early group indicates students who submitted their assignments on the lecture upload day. Intermediate group signifies students who submitted their assignments between the 2nd day and the day before the deadline. Late group indicates students who submitted their assignments on the deadline day. In Table 2A, the values in parentheses indicate P-values. In Table 2B, the covariates were age at examination and sex. The values are rounded up to the third decimal place and displayed to the second place.

4.  Ethical considerations

This study was approved by the Institutional Review Board of Sojo University (Permit Number: 2020-5). The study content was posted on the bulletin board for third-year university students. Students were allowed to opt-out of study participation, and we obtained informed consent for them by opt-out. This study was conducted in accordance with the Declaration of Helsinki.

Results

Table 1 presents the characteristics of the 139 pharmacy students. The median values for examination score and GPA were 70 points and 2.38, respectively. Regarding the time lag between the two homework assignments for each lecture (Supplementary material S1), the variation among the lectures was small. The homework assignment score for each lecture showed the same tendency (Supplementary material S1). Most students submitted their assignments for each lecture between 09:00 and 22:00 (Supplementary material S1). However, the number of hours between lecture upload and assignment submission varied considerably (Supplementary material S1). About 60% of students submitted their assignments on the day of lecture upload or on the deadline day (Supplementary material S2). Therefore, we categorized the students into three subgroups (early group, n = 42; intermediate group, n = 58; late group, n = 39) based on the number of hours from uploading a lecture to when assignment 2 was submitted. Among the three groups, the early and intermediate groups submitted their assignments at an earlier time of day than the late group (Table 1). Examination score and GPA in the former two groups were also higher than in the late group (Table 1). However, there were no differences in the time lag between the two assignments and total homework assignment score (Table 1). For each group, to investigate the degree of relationship between the variables, we examined Spearman’s rank correlation coefficients (Table 2A and Supplementary material S3). We confirmed a negative correlation between examination score and late time of day for assignment submission in the early group (Table 2A and Supplementary material S3a). By contrast, there was no such correlation in the intermediate and late groups (Table 2A and Supplementary material S3b, c). Next, we undertook multivariable regression analysis using examination score as the objective variable to investigate the relationship between examination score and submitted time of day of assignment without the influence of total homework assignment score and GPA (Table 2B). We found that examination score was negatively influenced by late submitted time of day of assignment in the early group (estimate, –0.02; P = 0.001) but not in the intermediate (estimate, 0.00; P = 0.44) and late groups (estimate, –0.01; P = 0.19).

Discussion

In the present study, we employed information related to when faculty students submitted their lecture-related homework assignments. We mainly investigated the relationship between examination score as academic performance and the submitted time of day of the assignments. One study reported the usefulness of the time information of access logs (e.g., opening homework, viewing, and submission date) related to homework assignments as an indicator of course achievement in on-demand education⁹⁾. That research employed the time information related to homework assignments to assess students’ procrastinating behavior. By contrast, we focused on the submitted time of day of the assignments.

In the early group, there was a positive relationship between examination score and total homework assignment score (Table 2A). That tendency was particularly prominent among students who submitted the assignment shortly after upload (Table 2A). This result indicates that such students effectively used the homework to understand the content of the lectures they had followed on schedule. Those students evidently displayed self-efficacy and time management as opposed to procrastination^6,8). By contrast, we observed a negative relationship between examination score and submitting the assignment late in the day (Table 2B). This negative relationship between examination score and late homework assignment submission was evident only in the early group (Table 2B). When the examination was over, we interviewed some students in the early group, who had worked late on their assignments on the 1st day. They stated they had watched the on-demand lecture and finished the assignments on that 1st day to relieve the psychological pressure and sense of duty of having to submit the assignments at some point. They may have been overly conscious about having to submit assignments on that 1st day rather than fully comprehending the lecture content: this illustrates a negative aspect with homework¹⁰⁾. It is possible that such students are unable to make good use of the flexibility offered with on-demand classes, which reflects a time management issue. It is therefore necessary to inform students that they should leave plenty of time to complete their homework assignments.

With the late group, the time of day when most students submitted their assignment for the on-demand classes was the afternoon of the deadline day (Sunday) (Table 1). With that group, both examination and total homework assignment scores were the lowest among all three groups (Table 1). We checked the access logs of lecture materials in WebClass and confirmed that about three quarters in the late group first accessed lecture materials on the deadline day. Thus, we believe their main focus was on submitting homework rather than on grasping the lecture content. These results further suggest that those students did not fully understand the content: they simply wanted to complete the assignment. These tendencies reflect academic procrastination—not academic achievement—and poor time management⁸⁾.

With the intermediate group, the students undertook the homework assignments at varying times compared with the other two groups (Supplementary material S3b). The examination score for the intermediate group was higher than that of the late group (Table 1). Furthermore, there was a positive relationship between examination score and GPA (Table 2). However, we discerned no correlation between examination score and time of day of assignment submission (Table 2). These results may indicate that some students in the intermediate group did their assignments by adjusting their schedules, and that led to good examination results.

On-demand education offers university students the opportunity to decide for themselves when they study. Therefore, it is necessary to analyze time factors and activities related to on-demand classes. When pharmacy students show inappropriate use of time, it may be important to communicate with them in that regard and proactively improve their time management. The present study has some limitations. First, it was performed at a university with a geographically limited sampling area for participants. Second, this was a retrospective study. A further prospective study is needed to assess the role of time of day when students work on assignments. Third, from the browsing history, we were unable to determine the total access time for each lecture. Fourth, there are possible confounding factors we did not assess that could affect both the examination score and time of day of assignment submission.

Our study revealed that some pharmacy students have poor time management for on-demand classes. Their time management may affect their understanding of class content.

Acknowledgments

We thank Edanz (https://jp.edanz.com/ac) for editing a draft of this manuscript.

There is no conflict of interest to be disclosed in relation to the contents of this paper.

Supplementary materials are included in the J-STAGE online version of this article.

References

• 1)   Yasuhara  T,  Kushihata  T,  Ueda  M, et al. Education for freshmen in the faculty of pharmacy in 2020—How to foster trust and a sense of belonging to the faculty? Japanese Journal of Pharmaceutical Education. 2021; 5: 2020-058. doi: 10.24489/jjphe.2020-058.
• 2)   Rasheed  AR,  Kamsin  A,  Abdullah  NA. Challenges in the online component of blended learning: a systematic review. Comput Educ. 2020; 144: 103701. doi: 10.1016/j.compedu.2019.103701.
• 3)   Pala  A,  Akyıldız  M,  Bağcı  C. Academic procrastination behaviour of pre-service teachers’ of Celal Bayar University. Procedia Soc Behav Sci. 2011; 29: 1418–1425. doi: 10.1016/j.sbspro.2011.11.381.
• 4)   Bandura  A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977; 84(2): 191–215. doi: 10.1037/0033-295x.84.2.191.
• 5)   Liu  G,  Cheng  G,  Hu  J, et al. Academic self-Efficacy and postgraduate procrastination: a moderated mediation model. Front Psychol. 2020; 11: 1752. doi: 10.3389/fpsyg.2020.01752.
• 6)   Kitsantas  A,  Zimmerman  BJ. College students’ homework and academic achievement: the mediating role of self-regulatory beliefs. Metacogn Learn. 2008; 4(2): 97–110. doi: 10.1007/s11409-008-9028-y.
• 7)   Katz  I,  Eilot  K,  Nevo  N. “I’ll do it later”: type of motivation, self-efficacy and homework procrastination. Motiv Emot. 2013; 38(1): 111–119. doi: 10.1007/s11031-013-9366-1.
• 8)   Núñez  JC,  Suárez  N,  Cerezo  R, et al. Homework and academic achievement across Spanish compulsory education. Educ Psychol (Lond). 2015; 35(6): 726–746. doi: 10.1080/01443410.2013.817537.
• 9)   Yang  Y,  Hooshyar  D,  Pedaste  M, et al. Predicting course achievement of university students based on their procrastination behaviour on Moodle. Soft Comput. 2020; 24(24): 18777–18793. doi: 10.1007/s00500-020-05110-4.
• 10)   Cooper  H,  Robinson  JC,  Patall  EA. Does homework improve academic achievement? A synthesis of research, 1987–2003. Rev Educ Res. 2016; 76(1): 1–62. doi: 10.3102/00346543076001001.