Class Report
The Effect of Multimodal Flashcards on L2 Derivational Vocabulary Knowledge:
A Preliminary Analysis of Attempts and Quiz Question Modes
2023 Volume 8 Pages 1-24
Details
2023 Volume 8 Pages 1-24
Many studies suggest learning vocabulary through derivational knowledge can be helpful in a second language (L2) but there may be notable differences between student comprehension and production of vocabulary based on derivational knowledge. Other research suggests that flashcards can be extremely useful for L2 vocabulary acquisition, while others suggest that practicing vocabulary in multiple modes (i.e., reading, speaking, writing) is beneficial to learners. However, there is less research regarding how studying derivational vocabulary through flashcards or multiple modes affects acquisition. This study seeks to offer a preliminary look at this problem and how study systems can be improved by reporting on the use of multimodal flashcards that provide multiple choice, writing, listening, and speaking activities to students learning derivational vocabulary. After using the flashcards, students took multimodal quizzes that contained multiple choice, listening, and writing questions. We checked for correlation between flashcard usage across multiple modes and quiz scores both overall and by mode to look for general associations, then performed multiple regression analyses with random forests to determine if certain modes were more effective at promoting acquisition. We found that flashcard usage generally correlates with higher quiz scores and that writing was the most impactful of the practice modes.
Derivational knowledge refers to the awareness, knowledge, and usage of roots and affixes across words families and in various contexts and mediums. In second language (L2) studies, it is often further specified as a learners’ understanding of the appropriateness of word parts in combination with a headword (i.e., Nation, 2013). However, it should be mentioned that this same skill is often referred to as morphological knowledge in first language (L1) studies (e.g., Carlisle, 2000; Iwaizumi & Webb, 2022). While there have been many studies regarding how derivational knowledge will affect word lists in L2 learning contexts and L2 vocabulary tests (e.g., Ishii & Schmidt, 2009; Mochizuki & Aizawa, 2000; Nation, 2013), fewer studies have focused on how to teach derivational knowledge and what impact those specific treatments will have on learners. In the realm of teaching, meaning-based flashcards have long been used to promote students’ memorization of vocabulary (e.g., Nakata, 2011; 2020; Schmitt & Schmitt, 1995) and multimodal learning, which encourages student interaction with the target knowledge in a variety of ways, is viewed positively for optimizing retention (e.g., Laufer & Hulstijn, 2001; Zarei & Khazaie, 2011). Since derivational knowledge is an important part of learning vocabulary, and learning vocabulary seems to be aided by multimodal learning, it follows that multimodal learning may also have a meaningful impact on derivational knowledge. However, few studies have quantified the learning effects of multiple modes on derivational acquisition and fewer still have sought to determine what impacts multiple modes of study will have on derivational knowledge in L2 vocabulary acquisition. Furthermore, as technology has improved, and multimodal learning systems are created, there is a need to verify the effectiveness of such systems and investigate what can be done to improve upon them. Therefore, our study seeks to examine what impact multiple modes of study via an online multimodal flashcard platform has on derivational vocabulary knowledge as demonstrated through multimodal vocabulary quizzes, with the hope of improving our own multimodal flashcard system and informing others looking to implement similar systems.
Derivational knowledge has been considered an important part of L2 vocabulary acquisition for many years (e.g., Ishii & Schmidt, 2009; Iwaizumi & Webb, 2022; 2023; Mochizuki & Aizawa, 2000; Nation, 1980; 2013; Snoder & Laufer, 2022). Within the Japanese English as a Foreign Language (EFL) context, Ishii and Schmidt (2009) showed that derivational knowledge was highly correlated to general vocabulary knowledge. Furthermore, Mochizuki and Aizawa (2000) have shown that Japanese learners’ knowledge of affixes contributed to overall vocabulary size, but that there were crucial gaps in their knowledge base. In contrast, works such as Wei (2015) suggest that knowledge of certain word roots for which the root meaning does not have an obvious connection to the word meaning (e.g., vers = turn, but converse = to talk) may not be associated with vocabulary knowledge. Furthermore, Iwaizumi and Webb (2022; 2023) have also suggested that there are differences in L2 learners’ ability to produce derivatives and their receptive knowledge of them. For example, they found that though L1 speakers were able to produce more derivatives in a decontextualized production task, L2 learners at the 3000-5000 vocabulary level produced a similar number of derivatives and Iwaizumi and Webb (2022) suggested this might be due in part to derivational vocabulary knowledge. They also found that productive ability was greatly linked to receptive ability, and that learners’ breadth of vocabulary was tied to their ability to accurately produce derivatives (Iwaizumi & Webb, 2023).
However, while several studies seems to suggest that poor knowledge of derivational affixes and roots hinders vocabulary size and reading skills (e.g., Collins & Nation, 2015; Hayashi & Murphy, 2011; Nation, 2020; McLean, 2018), there is less evidence to suggest that deliberate learning of these specific word parts leads to enhanced vocabulary knowledge or acquisition. Furthermore, it is still unclear how much vocabulary knowledge is gained on an item-based level (i.e., each unit memorized as a separate item) or a system-based level (i.e., units remembered based in part on previous knowledge and recognition of similar derivational forms) (e.g., Boers, 2021; Iwaizumi & Webb, 2023). Thus, it is also unknown how well systematic learning of derivational vocabulary, specifically, would aid in vocabulary acquisition, or if derivational vocabulary can even be learned in a systematic way. Thus, while acquisition of derivational affixes seems to be important for EFL learners, there is still much that is unknown about the best way to teach this knowledge and what type of practice will increase productive and receptive derivational abilities, if at all.
2.2 Multimodal Vocabulary LearningMultimodal learning is sometimes used interchangeably with multimedia-based learning but a distinction in academic circles is starting to form (Lauer, 2009). While both words contain the Latin root meaning “many,” media typically refers to ways of public distribution or means of publication, while modal was defined by Kress and van Leeuwen (2001) as the means that people interact with language, such as through the writing in books, speaking with others, listening to audio clips, or watching the word being acted out on screens. Specifically, in this study, we use the phrase multimodal vocabulary learning to indicate learning vocabulary through a variety of modes, and not necessarily through multiple modes at once in a single activity.
It has been suggested that multimodal vocabulary learning has advantages over learning vocabulary through a single mode (e.g., only through reading). Laufer (2017), for example, argues that the more details that a learner focuses on, the more likely they are to remember the target item. Furthermore, works such as Swain (1985) suggest that adding not only receptive tasks in vocabulary learning, but also productive tasks, such as speaking and writing, will increase the likelihood of remembering a word. Furthermore, Nation (2020) recommends using linked skills activities in order to increase learner uptake by having them interact with target items across different skills to “encourage repetition, retrieval, and varied meetings and use” (p. 24). Thus, while there is not a singular framework that indicates exactly which modes should be used in studying or how often each should be represented to increase vocabulary acquisition, it seems that the general body of research encourages interaction with vocabulary in a variety of ways. However, it should be noted that there is still no research that indicates how well this will translate to the learning not of specific vocabulary, but of specific derivative affixes. Furthermore, it is unclear if computer-assisted multimodal learning will have the intended impact on students and the extent to which various practice has cross-modal effects.
2.3 Learning with FlashcardsFlashcards are traditionally viewed as stacks of dual-sided, decontextualized vocabulary cards, typically organized by theme or textbook unit, which learners flip over to practice retrieval of pair-associated word forms and meanings. Because of their small size and paper medium, vocabulary units are often learned in isolation rather than being derived or integrated into varied contexts (Barcroft, 2020). However, computer programs and phone applications have become increasingly ubiquitous and have allowed for greater contextualization and learning modes. Technology-based flashcards allow for not only paired-retrieval, but also multiple modes of practice, such as contextualized multiple-choice questions, listening, or type-to-answer questions (Nakata, 2020). Such technologically-infused flashcards have been shown to aid in L2 vocabulary acquisition (e.g., Nakata, 2011; 2020) and found to improve motivation more than traditional paper flashcards (e.g., Ying, et al., 2021).
Takeda (2023) reported about the initial implementation of a website containing multimodal flashcards (i.e., flashcards with multiple modes of practice) to study a number of different vocabulary items at Tohoku University. The website provided students with flashcards that offered multiple-choice, writing, listening, and speaking questions. However, her study focused on the implementation of the system, which at the time was not able to log student information, and thus her report attempted to verify the efficacy of the flashcards through student self-reported usage, taken in a survey. Takeda (2023) found that students found the multimodal flashcards helpful in studying and their responses to survey questions seemed to show weak correlation to higher exam scores. However, without objective data on student usage attempts with the multimodal flashcards, the results were preliminary at best, and left open the question regarding exactly how much each mode of practice affected vocabulary retention. Additionally, her initial results suggested that students found the flashcards more helpful for learning non-derivational vocabulary items. Thus, it is still unknown to what degree multimodal flashcards usage or specific mode usage will most affect the acquisition of derivational vocabulary.
2.4 Research QuestionsBased on the aforementioned studies, learning derivational vocabulary can help L2 learners to expand their vocabularies and improve reading skills. Additionally, the previously mentioned studies suggest that exposure to vocabulary through multiple modes and with technology-infused flashcards are an effective method for L2 vocabulary acquisition of non-derivational vocabulary. However, fewer studies have looked at how technology-infused flashcards with multiple modes affect L2 derivational vocabulary learning. Furthermore, previous studies have still not made it clear how the rate or mode of practice will influence learners’ acquisition of either derivational-vocabulary or non-derivational vocabulary or their ability to apply the information in various modes and contexts. In order to provide a more in-depth look at the relationship between mode practice type, practice amount, and acquisition of derivational vocabulary, and to glean information that can help us to improve upon our implementation of multimodal flashcards, we attempt to answer the following research questions:
214 students at Tohoku University participated in this study, but only 154 provided informed consent and completed all activities, and thus the scores of 154 students were used for this study. They were taking an EFL class focused on improving English reading and vocabulary-building skills, as outlined in their textbook, Pathways to Academic English, 4th Edition. The students were mostly around CEFR B1 level, as judged by their TOEFL ITP® scores (400-607; M = 505.2; SD = 42.04) and based on the CEFR conversion chart offered by Educational Test Service (ETS, 2023). The TOEFL ITP® test was taken within a few weeks of this study.
3.2 Multimodal Flashcard CreationThe authors created a website (www.pathwaystoacademicenglish.com) with a login system that allows students to access a range of multimodal flashcards and records student attempts of their usage. The flashcards were created in JavaScript and PHP for the website and contain a number of different modes of study. Specifically, the modes consisted of multiple-choice meaning questions (MC-M), multiple-choice context (MC-C) questions, fill-in-the-blank writing (W) questions and speaking and listening practice provided by local Text to Speech (TTS) Application Programming Interface (API) for listening and local Automatic Speech Recognition (ASR) API for speaking practice. The flashcards were created in this manner in order to encourage retrieval to increase uptake, rather than simple presentation, following works such as Karpicke and Roediger (2008) and Nakata (2011; 2020). Students were able to select the words they wanted to practice freely, and also had the option to practice pre-packaged study sets to prepare for quizzes. Upon submitting a guess, multiple-choice and writing questions alert users as to whether or not their answer is correct, and the writing questions also contain a button to see the correct answer. Listening practice consists of a selector for either American or British English so that users can hear a sample phrase containing the target item in a range of pronunciations. Speaking practice activates local ASR for three seconds, during which time users can attempt to say the listening practice phrase. It then compares the final recognition text with the target phrase and checks if the recognition text contains the target item. If it does, then the user's pronunciation is considered intelligible (e.g., Spring & Tabuchi, 2021) and the user is alerted that they did satisfactorily. If the target item is not contained in the final recognition text, users are encouraged to try again. An example of each question type is displayed in Appendix 1.
The students’ textbook, Pathways to Academic English, 4th Edition, contains 25 different prefixes (e.g., pre-, re-, sub-), 55 pairs of Latin and Greek-based roots (e.g., hydro, bio, macro), and 30 grammatical suffixes (e.g., -ous, -ize, -tion), which the students are expected to study. These items are collectively called word parts in the students’ textbook and represent the base of the L2 derivational knowledge monitored in this study. The flashcards contained one of each type of question for each item, i.e., 110 multiple-choice meaning questions (one for each prefix, root, and suffix), 110 multiple-choice context questions, 110 writing questions, and 110 speaking and listening questions.
3.3 Classroom ProcedureAll students participating in the study were instructed by one of the two authors who taught from a unified curriculum, and thus used the same textbook, teaching materials, and homework assignments over the course of the study. For three classes, students used the textbook wordlists and the university’s common, corresponding worksheets for contextualized practice. Furthermore, after each class, students were also given the university’s common, corresponding homework assignments. However, the participants in this study were also asked to practice with the online flashcards outside of class. However, which exact vocabulary items and modes, and how much or often to practice with the flashcards was left to the discretion of the students. The website was created in such a way that teachers could download records of their students’ flashcard usage, and the authors of this study utilized this system to record student data. Table 1 displays the descriptive data of student flash card usage. As suggested by Table 1, multiple-choice questions were by far the most practiced mode, followed by writing, and then speaking and listening.
| Practice Mode | M | Mdn | SD | Range |
|---|---|---|---|---|
| Multiple-choice | 379.1 | 254 | 423.2 | 6-3434 |
| Writing | 136.9 | 84 | 162.8 | 0-1182 |
| Listening | 54.7 | 17 | 108.4 | 0-774 |
| Speaking | 80.1 | 18 | 125.1 | 0-768 |
| Total | 420.3 | 442.5 | 654.0 | 6-4704 |
Each week, after studying a particular set of derivational affixes, completing an online homework assignment, and conducting flashcard practice, students were given a multimodal vocabulary quiz. This procedure was continued for five sessions, meaning that there were five quizzes in total. The quizzes were created cumulatively, which means that items that were tested on the first quiz were also tested on the second quiz, in addition to new items. For example, the first quiz focused on the 25 prefixes, and only included questions about the 25 prefixes, but on the second quiz, which focused on the first 20 roots, 70% of the questions were about the first 20 roots while the remaining 30% were related to the 25 prefixes. On the third quiz, which focused on the second 20 roots, 70% of the questions were about the second 20 roots, and 30% were related to the 25 prefixes and the first 20 roots. We decided to use cumulative quizzes following works such as Nakata et al. (2021) that suggest repetition of vocabulary items increases retention. The quiz questions represented a range of modes, i.e., there were listening questions, writing questions, and both meaning and form based multiple-choice questions. We utilized a range of modes when creating the questions based on studies such as Uchihara (2022), which suggest adding extra modes to quizzes will encourage students to practice multiple modes and increase the ability of learners to recall them in a range of modes and contexts. Listening questions asked student to listen to a phrase and then select the word part that they heard from a list. Multiple-choice questions consisted of both meaning-based and context-based questions, similar to those found on the flashcards, albeit with different questions. Writing questions consisted of a fill-in-the-blank question with half of a word given and the target derivational affix missing. Students received full credit for either correctly rewriting the entire word or writing just the missing affix. Table 2 denotes the descriptive data of students’ average quiz scores by mode type. The data for quiz questions seem to be generally normal, albeit with some underperformers lowering the total average. Furthermore, the data shows that writing questions seemed to be the most difficult, which is perhaps due to them being the only productive question type.
| Question Mode | M | Mdn | SD | Range |
|---|---|---|---|---|
| Listening | 0.8630 | 0.9 | 0.0987 | 0.45-1 |
| Multiple-choice | 0.8771 | 0.9 | 0.0857 | 0.4-1 |
| Writing | 0.7029 | 0.7 | 0.1392 | 0.4-1 |
| Total | 0.8429 | 0.85 | 0.0859 | 0.2-.95 |
In order to determine the impact of the multimodal flashcards on quiz scores in general, we first conducted correlation analysis between all pairs of quiz scores and flashcard mode attempts. We checked for trends in the data to see if there seemed to be any general associations between particular modes. Next, we conducted multiple regression analyses with the quiz scores as the dependent variable. For predictor variables, we included TOEFL-ITP® scores as a way to account for previous knowledge and general L2 proficiency, and conducted a stepwise model for the other variables, i.e., all of the various data from the multimodal flashcards. Our stepwise model was created by checking that each potential predictor variable: (1) was individually statistically significantly correlated to the dependent variable, using cutoffs of r > .1 and p < .05, and (2) there was no multicollinearity amongst predictor variables above r >= .700. Since (1) was true for all variables, and there were no cases of (2) for any of the modes or between modes and TOEFL-ITP® scores, all individual flashcard usage modes were included in the model. We also provided VIF scores for each variable to ensure there was not an abundance of multicollinearity. Following Mizumoto (2023), relative importance and contribution of variables was tested using dominance analysis rescaled as relative weights, and random forests were used to confirm or reject whether or not the variable had any discernable impact (rather than the p values). The decision to use the random forests for confirmation or rejection was influenced not only by Mizumoto (2023), but by the fact that some flashcard usage exhibited skewed data and that the suggestion by the American Statistical Association that p-values alone do not provide a good measure for accepting or rejecting a hypothesis (Wasserstein & Lazar, 2016). Regression models were created using the total average quiz scores, and then repeated using the average scores of each quiz question mode to see if certain modes of flashcard practice were more useful for certain modes of quiz questions.
Table 3 shows the correlation matrix between flashcard practice attempts by mode and quiz scores by mode. All correlations in Table 3 were statistically significant (p < .05), with the exception of the correlation between multiple-choice flashcard usage and quiz listening score questions. The results of this table also show that multiple-choice quiz questions had the largest impact on overall quiz scores. Furthermore, there appears to be a trend that usage of other flashcard modes exhibited stronger correlation to various quiz question types than the multiple-choice question mode.
| Q_L | 0.675 | |||||||
| Q_MC | 0.926 | 0.496 | ||||||
| Q_W | 0.774 | 0.427 | 0.564 | |||||
| FC_MC | 0.238 | 0.119 | 0.224 | 0.242 | ||||
| FC_W | 0.489 | 0.359 | 0.422 | 0.469 | 0.530 | |||
| FC_L | 0.312 | 0.242 | 0.285 | 0.245 | 0.387 | 0.553 | ||
| FC_S | 0.376 | 0.239 | 0.340 | 0.324 | 0.269 | 0.547 | 0.583 | |
| FC_Tot | 0.399 | 0.252 | 0.363 | 0.376 | 0.895 | 0.788 | 0.665 | 0.598 |
| Q_Tot | Q_L | Q_MC | Q_W | FC_MC | FC_W | FC_L | FC_S |
Table 4 shows the results of the regression analysis for overall quiz scores as predicted by TOEFL ITP® scores and individual flashcard mode practice types, which predicted 36.2% of the variance; R2 = .362, F = 16.00, p < .001. Figure 1 shows the results of the random forest analysis that compliments the regression analysis. Overall, these results suggest that after correcting for general academic English ability, as indicated by the TOEFL ITP® scores, most of the flashcard modes offered some modicum of contribution to the model, but that the number of writing attempts was by far the type of practice that was most impactful on quiz scores overall.
| I.V. | B | S.E. | β | p | VIF | R.W. | R.F. |
|---|---|---|---|---|---|---|---|
| TOEFL | 7.2e-4 | 1.46e-4 | .354 | .000 | 1.20 | .17 (46%) | Con. |
| FC_MC | -1.1e-5 | 1.55e-5 | -.055 | .469 | 1.42 | .01 (4%) | Tent. |
| FC_W | 1.6e-4 | 4.82e-5 | .306 | .001 | 2.01 | .11 (30%) | Con. |
| FC_L | 3.1e-5 | 6.71e-5 | .039 | .646 | 1.76 | .03 (8%) | Con. |
| FC_S | 4.4e-5 | 5.82e-5 | .064 | .453 | 1.75 | .04 (12%) | Con. |
Table 5 shows the results of the regression analysis for quiz listening question type scores, which predicted 14.7% of the variance; R2 = .147, F = 4.87, p < .001. Figure 2 shows the results of the random forest analysis that compliments the regression analysis. Although these variables predicted quiz listening scores much more poorly than overall quiz scores, a similar pattern emerged in that the random forest confirmed all variables except for the multiple-choice flashcard usage, and the greatest predictor amongst the flashcard mode usage was the writing mode.
| I.V. | B | S.E. | β | p | VIF | R.W. | R.F. |
|---|---|---|---|---|---|---|---|
| TOEFL | 3.0e-4 | 2.01e-4 | .128 | .138 | 1.20 | .03 (20%) | Con. |
| FC_MC | -2.8e-5 | 2.13e-5 | -.123 | .186 | 1.42 | .01 (4%) | Rej. |
| FC_W | 1.9e-4 | 6.36e-5 | .322 | .004 | 2.01 | .07 (50%) | Con. |
| FC_L | 7.4e-5 | 9.24e-5 | .081 | .428 | 1.76 | .02 (15%) | Con. |
| FC_S | -1.9e-5 | 8.02e-5 | -.002 | .570 | 1.75 | .02 (11%) | Con. |
Table 6 shows the results of the regression analysis for quiz multiple choice question type scores, which predicted 26.4% of the variance; R2 = .264, F = 10.10, p < .001. Figure 3 shows the results of the random forest analysis that compliments the regression analysis. These variables predicted multiple choice questions very similarly to the overall quiz scores, which is likely due to the fact that the multiple-choice question scores were highly correlated with overall quiz scores. The general trend of the writing practice mode being the most impactful on the specific quiz question mode continued for multiple choice quiz questions, as with overall scores and listening quiz questions.
| I.V. | B | S.E. | β | p | VIF | R.W. | R.F. |
|---|---|---|---|---|---|---|---|
| TOEFL | 6.1e-4 | 1.57e-4 | .308 | .000 | 1.20 | .12 (45%) | Con. |
| FC_MC | -4.3e-5 | 1.67e-5 | -.022 | .799 | 1.42 | .01 (5%) | Rej. |
| FC_W | 1.2e-4 | 5.18e-5 | .242 | .020 | 2.01 | .07 (28%) | Con. |
| FC_L | 3.5e-5 | 7.21e-5 | .047 | .627 | 1.76 | .03 (9%) | Con. |
| FC_S | 5.0e-5 | 6.23e-5 | .076 | .430 | 1.75 | .04 (14%) | Con. |
Table 7 shows the results of the regression analysis for quiz writing question type scores, which predicted 35.9% of the variance; R2 = .359, F = 15.81, p < .001. Figure 4 shows the results of the random forest analysis that compliments the regression analysis. The same trend continued for this mode type as with other types, although for writing questions, the random forest did not confirm the impact of using the flashcard listening mode.
| I.V. | B | S.E. | β | p | VIF | R.W. | R.F. |
|---|---|---|---|---|---|---|---|
| TOEFL | 7.3e-3 | 2.45e-4 | .400 | .000 | 1.20 | .19 (52%) | Con. |
| FC_MC | -9.7e-6 | 2.60e-5 | -.030 | .710 | 1.42 | .02 (5%) | Rej. |
| FC_W | 2.8e-4 | 8.10e-5 | .330 | .001 | 2.01 | .11 (30%) | Con. |
| FC_L | -4.0e-5 | 1.13e-4 | .047 | .726 | 1.76 | .02 (4%) | Tent. |
| FC_S | 4.1e-5 | 9.78e-5 | .076 | .674 | 1.75 | .03 (9%) | Con. |
With regards to the first research question, it seems that flashcard usage is generally associated with higher quiz scores. The results of Table 3 show that overall flashcard usage is correlated with higher total quiz scores, albeit weakly, and that various modes of flashcard usage are also correlated with various modes of quiz questions, with the exception of multiple-choice flash card usage and listening quiz questions. Furthermore, the various multiple regression models suggest that when correcting for previous knowledge by including TOEFL ITP® scores, most of the modes of flashcard practice had some discernable impact on quiz scores. Although there is variance in the amount of correlation between various flashcard modes and quiz scores, and how much each flashcard mode contributed to each regression model, there does seem to be a general trend that practicing with the flashcards tended to be associated with higher quiz scores, and thus likely had at least a small amount of impact on student learning of derivational affixes.
The aforementioned findings have some similarities with previous research on vocabulary acquisition, which has emphasized the benefits of repeated exposure and retrieval practice (e.g., Karpicke & Roediger, 2008; Nakata, 2011; 2020). Specifically, there is a pattern of more practice being generally tied to better final results, which itself, is not particularly surprising. However, there are a few discrepancies between this study and those of previous studies that focus on non-derivational vocabulary. Specifically, multiple-choice question flashcard practice alone did not seem to have nearly as much impact on learning as writing question flashcard practice. This could be due in part to the fact that derivational vocabulary learning requires that learners apply systematic knowledge. According to the revised taxonomy of thinking skills (Anderson et al., 2001), application is a higher order skill than memorization, which is generally used for non-derivational vocabulary learning. Therefore, production practice might be much more important for developing the skill to apply derivational knowledge to vocabulary items. Another possibility is that the multiple-choice questions in this study did not lead to enough retrieval practice, as the choices were shown at the same time as the questions, and van den Broek et al. (2023) have recently shown that delaying the choices in multiple-choice exercises can increase retrieval and final acquisition. However, further study and direct one-to-one comparisons between students’ acquisition of derivational and non-derivational vocabulary would be required to know for sure which explanation is more appropriate.
As for the second research question, we found that some practice modes had greater impact on quiz scores than others, and that there was some effect of quiz question mode. First, there was a trend in the correlation matrix (Table 3) for flashcard writing question practice to exhibit the strongest amount of correlation to quiz scores, followed by speaking and listening question practice, which were comparable, followed by multiple choice question practice, which had the weakest amount of correlation. This general trend also continued across quiz question mode types. Furthermore, when checked with the regression analyses and random forests, after correcting for general L2 ability by including TOEFL ITP® scores in the model, writing question practice was shown to be the most impactful flashcard practice mode, and this trend was found for overall quiz scores as well as for each quiz question mode. It should be noted that speaking and listening question practice modes were also confirmed as impactful in each random forest analysis, although these analyses and the corresponding regression analyses clearly show that these practice modes were much less impactful than writing questions. Importantly, though, the gap between how impactful writing practice was versus speaking and listening practice was much less pronounced for listening quiz questions. Comprehensively, these results suggest that flashcard writing practice had the greatest impact on students’ quiz scores, that speaking and listening practice had a small impact on quiz scores, and that multiple choice question practice had negligible impact on quiz scores.
The results of the previous paragraph might be explained partially by the data set. Specifically, the descriptive data regarding flashcard usage in Table 1 suggests quite a bit of deviation from the median and average usage, indicating that there were several students who practiced a great deal more than the typical students, and there were also several who likely did not practice any of the modes very much. Furthermore, though every student practiced at least a little bit with multiple-choice flashcard questions, there were some students who did no writing practice at all (N = 16), and more who did not do any listening (N = 45) or speaking (N = 53) practice at all. Of the three less popular modes, writing was the most popular, with students overall doing less listening and speaking practice in general, and therefore, the trend in our data set for writing practice to appear to be the most impactful might be due less to the specific mode of writing, and actually be showing an association between quiz scores and “practice beyond multiple-choice questions” or varied mode practice. This explanation would also help to explain why speaking and listening practice modes were confirmed by the random forest analyses but did not appear to be more impactful in the regression analyses. Similarly, the fact that the multiple-choice practice did not show a stronger association with quiz scores could be due to the fact that this includes several students who used the multiple-choice questions exclusively but did not use multiple practice modes. Such an interpretation would match previous studies that have shown empirically or suggested theoretically that multiple practice modes bolster L2 vocabulary acquisition (e.g., Laufer, 2017; Laufer & Hulstijn, 2001; Nation, 2020; Schmidt, 2000). However, as mentioned above, it is also possible that productive practice is more important for derivational vocabulary learning than non-derivational vocabulary learning because the former requires more application of conceptual knowledge than non-derivational vocabulary learning, which is largely accomplished through memorization – a lower order thinking skill (Anderson et al., 2001).
It should also be noted that the results do seem to show some effect of specific practice type on specific practice mode. Specifically, listening and speaking practice had a more significant impact on listening quiz questions than on multiple-choice or writing quiz questions, which is somewhat unsurprising because both are skills which engage students’ orally and therefore could theoretically aide the ability to listen to the target items later (e.g., Uchihara, 2022). Furthermore, it is notable that speaking practice had a greater impact on writing questions than listening practice, which could be due to the fact that speaking and writing are both productive skills that require much more memory retrieval (e.g., Iwaizumi & Webb, 2023). These results also suggest that practicing in multiple modes likely has not only domain specific effects (i.e., practicing listening is related to being able to listen), but cross-domain effects (i.e., practicing productive skills helps productive skills, regardless of whether or not they are aural skills, and practicing oral skills helps oral skills, regardless of whether or not they are productive).
The insights gleaned from the results and considerations above contribute to our understanding of how to promote the learning of derivational vocabulary in our specific Japanese EFL context but can also be considered in a broader L2 learning context. Specifically, it seems that there is some merit in making sure that students are engaging in multimodal practice when acquiring derivational vocabulary, and that writing practice seems to be particularly important. Furthermore, our results suggest that multimodal flashcards such as those described in this study are one way to provide such practice, but that teachers should likely find ways to encourage students to use various modes of practice. By doing so, learners have critical opportunities for repeated exposure, retrieval practice, and contextual learning in multiple modes, which has long been suggested to aide L2 vocabulary acquisition (Laufer, 2017; Laufer & Hulstijn, 2001; Schmidt, 2000), and is underscored by the results of this study.
Finally, it should be mentioned that this study suffers from several limitations. First, at the current time, we are unable to discern how many attempts students made at each of the two types of flashcard multiple-choice practice questions (i.e., the context-based questions and the meaning-based questions) and can only see the number of attempts at multiple-choice questions in general. There might have been differences in effect if students primarily used meaning-based or context-based questions, but we are currently unsure of this. Second, we do not have a more formal way of determining change in vocabulary knowledge in this study and instead rely on using students’ in-class quiz scores. In the future, it would be beneficial to see if there is any discernable effect of flashcard-based practice on changes in standardized tests, such as the TOEFL ITP®. Additionally, there is the possibility that writing flashcard practice, specifically, had the greatest impact on scores in our study due to the fact that derivational vocabulary learning requires the application of systematic knowledge, and therefore, it is still unknown how congruent our results are with those of non-derivational vocabulary learned in the same way. Therefore, future studies should aim to repeat this experiment with both types of vocabulary to elucidate the impact of vocabulary type (i.e., derivational versus non-derivational) on the amount of vocabulary acquisition that can be acquired via multiple modes. Finally, when more data is available, it may be beneficial to our model to attempt to transform flashcard attempt data to make the regression models more meaningful. However, despite these limitations, we feel that we were still able to draw some preliminary conclusions that provided us with important clues regarding how to improve our flashcard system and implement it with students in the future.
Based on the results of this study and how it fits into the previous studies, as discussed above, we conclude that the multimodal flashcards did have some impact on quiz scores, but that learners who utilized multiple modes, especially the writing mode, were more likely to have higher quiz scores. This suggests that engaging with multiple modes aides L2 learners’ derivational vocabulary acquisition and that multimodal flashcards is one potential way of accomplishing this. Therefore, we intend to encourage students to engage in a wider variety of practice modes and to continue to take data to monitor the effects of multimodal practice on vocabulary acquisition using the flashcards in our particular context. We acknowledge that the study is limited in that it does not employ a true pre-posttest design with a standardized test and we did not have access to some fine-grained data. However, we think it contributes to existing literature by providing a preliminary look at how multiple modes of interaction with L2 derivational vocabulary can affect learning and can offer an important piece of advice to EFL teachers who are teaching word parts or other derivational vocabulary: encourage students to engage with the vocabulary in multiple modes, especially productive modes.
This paper was made possible in part by an Educational Resources Development Grant provided by Tohoku University. The authors would like to thank the editors of this publication and the advice of two blind reviewers for their help with drafting this manuscript.
