Research Note
A Review of Constructs of Critical Thinking Skills
2023 Volume 8 Pages 25-41
Details
2023 Volume 8 Pages 25-41
To contend with today’s rapidly advancing information society, critical thinking (CT) skills have been recognized as significant not only in the native language but also in contexts where English is learned as a foreign language (EFL). However, previous studies have not provided a consistent definition of the CT skill construct. Therefore, the present study aimed to organize the constructs of CT skills proposed in previous CT research. Specifically, this study investigated similarities and differences in these constructs based on the theoretical background of CT and articles on the development of CT tests. The results indicated that CT skills comprise four skills: (a) identifying main points, (b) examining content, (c) inferring conclusions, and (d) metacognizing one’s own thoughts. Furthermore, specific characteristics of test items for measuring these CT skills were also established based on previous studies. The findings of this review provide suggestions for future research on the cultivation and assessment of CT skills in an EFL context.
Since information technology has provided us with access to large amounts of information, we need to analyze and evaluate the information carefully. Thus, in today’s information society, it is essential to cultivate thinking skills to (a) examine the information closely, (b) create persuasive opinions based on appropriate evidence, and (c) reflect on one’s own thinking process from multiple perspectives (Din, 2020; Fahim et al., 2010; Varaporn & Sitthitikul, 2019). These thinking skills include critical thinking (CT) skills as one of the subcomponents because CT is a skill for identifying and examining information with skepticism to infer unbiased conclusions and understand the content accurately (e.g., Ennis, 1987). Therefore, CT has been regarded as an important aspect in the 21st century (ATC21S, 2022). In addition, due to globalization, there is an increasing opportunity to access information written in English even in environments where English is learned as a foreign language (EFL), such as Japan. Since EFL learners are also expected to perform CT skills through English, some previous studies attempt to evaluate these skills in an EFL context (e.g., Hirai et al., 2022). However, the lack of agreement in the CT construct among previous studies or raters makes the evaluation of CT skills challenging (Mehta & Al-Mahrooqi, 2015; Sato, 2022; Stapleton, 2001). To address this gap, the present study aimed to provide a guideline for the assessment and evaluation of CT skills in an EFL context by reviewing the CT constructs proposed in previous studies.
1.1 Definitions of CT SkillsCT has been defined in several previous studies. For example, CT has been defined as “reasonable reflective thinking focused on deciding what to believe or do” (Ennis, 1987, p. 10). Another study has described CT as a purposeful and self-regulatory judgment that objectively examines the content and one’s own thinking process (Facione, 1990, p. 3). Following these studies, Kusumi and Michita (2016) have defined CT as follows: (a) logical and unbiased thinking based on evidence, (b) reflective and thoughtful thinking that consciously examines one’s own thought processes, and (c) purposeful thinking that is performed in response to goals and contexts to think better.
Despite the variations in definitions of CT, certain commonalities can be observed. Summarizing these previous studies, it could be defined that CT is not a form of thinking that denies things, but thinking that does not accept information without questioning it and judges things based on the reflection of one’s own thought process.
1.2 Previous Research on the Assessment of CT Skills in an EFL ContextWith reference to previous studies on the theoretical background of CT, some empirical studies on the development of multiple-choice English tests to measure CT skills have been conducted. For example, Hirai et al. (2022) developed a multiple-choice English CT test (ECTT) for Japanese EFL learners. The study established the following three target domains: consistency—the skills of clarifying the logical structures, analysis—the skills of analyzing the relevant necessary information, and inference—the skills of deductive and inductive reasoning. This construct was validated using exploratory factor analysis and structural equation modeling. The results indicated that this construct fulfilled its validity to some extent and suggested insightful implications for the development of CT tests for EFL learners. Although the construct of the ECTT was designed with reference to previous CT tests developed for native English speakers (i.e., Facione & Facione, 1990), it might be necessary to further re-examine this construct because the test items were created based on the English Proficiency Test and the Synthetic Personality Inventory (SPI) test (Hirai et al., 2020). In other words, it appears that the construct of CT tests should be examined with reference to CT test items in previous studies.
Kubo (2023) also developed a multiple-choice CT test for English education (CTTEE) by referencing previous studies on CT tests for native English speakers (i.e., Ennis et al., 2005; Watson & Glaser, 2002) and validated the CT construct through exploratory factor analysis. Similar to previous studies (Ennis, 1987, 2018), the following three target domains were established: clarification—the skills of identifying main points, basis of inference—the skills of examining the content, and inference—the skills of inferring conclusions. Although many test items were deleted because they did not adequately meet item discrimination criteria, the results showed construct validity and reliability of the CTTEE.
Moreover, it has been stated that CT and argumentative writing are closely related (e.g., Hirvela, 2017; Mehta & Al-Mahrooqi, 2015). Specifically, argumentative writing requires not only producing claims and evidence, but also reflecting on the organization and content of the text, which encourages learners’ CT skills (Chason et al., 2017; Sato, 2022). Thus, some previous studies (e.g., Qoura & Zahran, 2018; Stapleton, 2001) have attempted to assess CT skills of EFL learners through writing tasks. However, these studies assessed CT skills by utilizing existing essay rubrics and writing models, which are not specific criteria for evaluating CT skills. This is because there are various interpretations of CT skills among previous research, making it difficult to capture the CT construct. Unlike these quantitative studies, Sato (2022) qualitatively analyzed an argumentative essay for Japanese EFL learners to investigate how raters assess CT skills and identify the characteristics of essays in which CT skills are performed. The results of verbal protocol showed that raters referred to the following four features when assessing CT skills of an essay: (a) relevancy—the degree of appropriate answering to the given questions; (b) content development—the degree of deeply discussing ideas with specific examples; (c) logicality—the logic and coherence of the essay; and (d) quality of ideas—the degree of validity or originality of the ideas in the essay. It is expected that these four features would contribute to the development of a rubric specialized for the evaluation of CT skills, however, this requires clarifying the construct of CT skills.
1.3 The Present StudyThe CT construct has been validated through the development of multiple-choice English CT tests (Hirai et al., 2022; Kubo, 2023). However, these CT tests only evaluated a subset of the CT construct in order to maintain test validity and reliability, and did not comprehensively validate the CT construct. In addition, since each CT test was based on different existing CT tests developed for native English speakers, the test constructs were slightly varied. Regarding the production-based task, CT assessment models have not been developed because the CT is ambiguous and not clearly defined (Sato, 2022). Nevertheless, there have been few attempts to comprehensively review the constructs of CT skills based on the theoretical background of CT in an EFL context. Therefore, clarifying the similarities and differences among the constructs described in previous studies would contribute to resolving this gap. To attain a further comprehensive understanding of CT skills, the present study aimed to organize the constructs of CT skills proposed by previous studies. Achieving this goal could provide important implications for the future assessment and cultivation of CT skills.
The constructs of CT skills are primarily described in articles that: (a) provide an overview of the theoretical background of CT; and (b) aim to develop CT tests for native English speakers. This study first reviewed articles that described the theoretical background of CT skills, followed by an overview of the constructs of CT skills in articles that developed CT tests. Through these steps, this study organized the construct of CT skills.
Regarding the theoretical background of CT skills, Ennis (1987, 2018) and Facione (1990, 2015) provided detailed descriptions and constructs of CT skills. As these articles are frequently cited in previous studies on CT (e.g., Chason et al., 2017; Sato, 2022), it could be said that Ennis and Facione have significantly impacted the field of CT research. Regarding CT test development, the following CT tests have been widely used among native English-speaking adults and college students: (a) the Cornell Critical Thinking Test (CCTT) (Ennis et al., 2005), (b) the California Critical Thinking Skills Test (CCTST) (Facione & Facione, 1990), and (c) the Watson-Glaser Critical Thinking Appraisal (WGCTA) (Watson & Glaser, 2002). These tests have also been cited in studies on the assessment of CT skills for EFL learners (e.g., Hirai et al., 2022; Kubo, 2023).
Thus, the present study reviewed the following articles with high impact factors in the field of CT research: (a) Ennis (1987, 2018), (b) Facione (1990, 2015), (c) Ennis et al. (2005), (d) Facione and Facione (1990), and (e) Watson and Glaser (2002). (a) and (b) proposed the constructs of CT skills by describing the theoretical background of CT skills in detail. (c), (d), and (e) proposed specific test items to measure the CT constructs. Note that although there are various definitions of CT skills, only the above articles form (a) to (e) specifically and clearly describe the CT constructs. Additionally, since Hirai et al. (2022) and Kubo (2023) examining the CT construct referred to the above-mentioned articles and selected a part of the CT construct to develop CT tests, they were not included in the present study.
As mentioned above, Ennis (1987, 2018) and Facione (1990, 2015) organized the constructs from a theoretical background and are frequently cited in previous studies on CT. Below, the constructs of CT skills proposed by each of the previous studies in terms of theoretical background are introduced.
Initially, Ennis (1987) defined the following four areas of CT skills: clarification, advanced clarification, basis of inference, and inference. Clarification is the skill of understanding content accurately and identifying its main points. Basis of inference is the skill of examining the content, and inference is the skill of drawing the correct conclusions and facts based on evidence. Clarification is the first step toward problem-solving and action decisions and is a complex and broad skill influenced by various thinking skills required for the CT process (Ennis, 2015; Kusumi & Michita, 2015). Thus, when the clarification is closely related to basis of inference and inference, it is distinguished as advanced clarification. In addition, Ennis defined the subcomponents of the above four CT areas to clarify the construct (see Table 1). The CT skill of understanding elementary graphs—a subcomponent of clarification presented in Table 1—is the only new subcomponent added by Ennis (2018). In fact, Ministry of Education, Culture, Sports, Science, and Technology (MEXT) (2016) also states the importance of the skill to understand graphs and tables. Similarly, high-stakes tests in Japan, such as the Common Test for University Admissions and Test of English for Academic Purposes (TEAP), include test items on statistical data and graphs. Consequently, understanding graphs written in English seems to be essential for Japanese EFL learners.
| Area | Subcomponent | Definition |
|---|---|---|
| Clarification | Focusing | Identifying main points such as a topic, a claim, and a conclusion |
| Analyzing argument | Organizing content using summaries and tables | |
| Understanding elementary graphs | ||
| Advanced clarification | Defining terms | Defining the meaning of ambiguous terms by inferring from the content |
| Identifying assumptions | Identifying implicit assumptions by examining the consistency of evidence and conclusions and inferring the writer’s implicit ideas and positions | |
| Basis of inference | Judging credibility | Judging the credibility of an information source by comparing and examining the content among multiple pieces of information |
| Observing | Observing and examining the content of information by checking the connection and consistency of evidence and conclusions | |
| Inference | Deducing & Inducing | Inferring correct conclusions and hypotheses based on the presented facts or from multiple pieces of information deductively and inductively |
| Value judgements | Making an unbiased decision by considering a variety of elements such as background factors and balance |
Moreover, Facione (1990) also formulated the construct of CT skills and presented the following six cores of CT skills: interpretation, analysis, evaluation, inference, explanation, and self-regulation. These CT skills are necessary for strong critical thinkers and enable purposeful reflective judgment with the combination of each CT core. Table 2 shows this construct of CT skills. Unlike Ennis (1987), although Facione did not present subcomponents, this article comprehensively described the construct by clarifying the six cores of CT skills. However, similar to Ennis (2018), Facione (2015) added the skill related to understanding graphs as one of the specific skills of interpretation.
| Core | Definition |
|---|---|
| Interpretation | Comprehending contents accurately EX) clarifying the meaning of ambiguous terms, distinguishing between important and unimportant information by organizing content, and understanding the contents of graphs |
| Analysis | Identifying the relationships among contents EX) identifying similarities and differences among multiple contents and clarifying their claims and evidence |
| Evaluation | Assessing the credibility of contents and logical strength of claims EX) evaluating the validity of the content, examining the credibility of sources and contents, and comparing the pros and cons of multiple pieces |
| Inference | Drawing reasonable conclusions based on evidence EX) identifying the evidence and arguments to draw reasonable conclusions from the information and formulating hypotheses based on the relevant information |
| Explanation | Presenting contents in a cogent and coherent way EX) explaining the process that leads to the conclusion logically and justifying the use of information for drawing the conclusion |
| Self-regulation | Monitoring their own cognitive process self-consciously EX) reflecting on the inferring by considering the influence of prejudices and personal ideas |
Tables 1 and 2 indicated that the names of the constructs proposed by Ennis (1987) differ from those presented by Facione (1990). Moreover, Facione (2015) focused more on production skills by establishing explanation, a core CT skill presented in Table 2. This skill promotes the development of logical and persuasive opinions. By contrast, Ennis (2018) focused more on reception skills by establishing detailed subcomponents of CT skills related to understanding the content of information (e.g., focusing, defining terms). However, it was determined that many compatible skills can be identified in each construct of CT skills by examining their definitions. For example, clarification in Table 1 corresponds with interpretation in Table 2, and basis of inference in Table 1 corresponds with analysis and evaluation in Table 2. Table 3 summarizes the common points of the constructs of CT skills between Tables 1 and 2.
| CT Skills Defined by Ennis |
CT Skills Defined by Facione |
Common Points |
|---|---|---|
| ・Focusing ・Analyzing argument ・Defining terms ・Identifying assumptions |
・Interpretation ・Analysis |
Identifying the main points and clarifying the contents |
| ・Judging credibility ・Observing |
・Evaluation ・Explanation |
Examining information closely for reliability and consistency |
| ・Deducing & Inducing | ・Inference ・Explanation |
Drawing the correct conclusion based on the facts and evidence |
| ・Value judgements | ・Self-regulation | Metacognizing one’s own thinking process |
To summarize the results of Table 3, the theoretical background of CT indicates that the construct of CT skills comprises: (a) identifying main points (the skills of accurately understanding the content of information to identify the main points), (b) examining content (the skills of examining the content of information), (c) inferring conclusions (the skills of drawing correct conclusions based on the information), and (d) metacognizing one’s own thoughts (the skills of reflecting one’s own cognitive process). Thus, the present study names the four constructs of CT skills as such.
3.2 Constructs of CT Skills in the Articles Developing CT TestsThe following three multiple-choice CT tests are widely used for native Enlgish spearkers: (a) the CCTT (Ennis et al., 2005), (b) the CCTST (Facione & Facione, 1990), and (c) the WGCTA (Waston & Glaser, 2002). In the process of developing CT tests, these studies defined the constructs of CT skills that the tests attempted to measure by refferring Ennis (1987) and Facione (1990). As these tests were examined with some validity and reliability, it seems that the constructs are worth referring to. Note that for the related CT skills in each table, the names of the construct of CT skill (based on previous research on the theoretical background of CT, i.e., identifying main points, examining content, inferring conclusions, and metacognizing one’s own thoughts) are used.
Firstly, The CCTT (Ennis et al., 2005) was developed based on the construct shown in Table 1. This CT test comprises the following six subscales: deduction, meaning & fallacies, observation & credibility of sources, induction, definition, and assumption identification. Table 4 summarizes the characteristics of the CCTT. As a result of item analysis, the reliability coefficients (Cronbach’s alpha) for each subscale ranged from .55 to .76, and the results of the split-half method confirmed the internal consistency of this CT test (r = .50–.77). Additionally, positive correlations with the WGCTA were also confirmed (r = .48–.79), which fulfills the criterion-related validity of the CCTT.
| Subscale | Related CT Skills | Item Characteristics |
|---|---|---|
| Deduction | Inferring conclusions | Judging whether a conclusion can be inferred from given facts |
| Meaning & Fallacies | Examining content | Identifying the reasons why the presented opinion is faulty |
| Observation & Credibility of sources | Examining content | Judging the most credible information from the presented material |
| Induction | Inferring conclusions | Judging whether given conclusions can be inferred from experimental results Determining the appropriate experimental design for testing given hypotheses |
| Definition | Identifying main points | Identifying the meaning of words that are perceived differently by various actors in a discussion |
| Assumption identification | Identifying main points Examining content |
Judging the validity of opinions in a discussion |
Secondly, The CCTST (Facione & Facione, 1990) was developed based on the construct shown in Table 2. This CT test comprises the following eight subscales: analysis, interpretation, evaluation, inference, explanation, induction, deduction, and numeracy (CCTST, n.d.). Table 5 summarizes the characteristics of the CCTST. As a result of correlation analysis, the internal consistency reliability of this test was confirmed (r = .68–.70) (Facione & Facione 1994, Facione 1997, Bowles, 2000). Further, it was revealed that the score of this test had a significant positive correlation with that of the WGCTA (r = .41, p < .001) (Facione, 2000), which confirms the criterion-related validity of the CCTST. However, it is challenging to determine the test items that measure the subscales of CT skills because there is no specification of which test items correspond to which subscales (Hirai et al., 2020; Reid, 2000).
| Subscale | Related CT Skills | Item Characteristics |
|---|---|---|
| Analysis | Identifying main points | Identifying assumptions, reasons, topics, and evidence in arguments and explanations |
| Interpretation | Identifying main points | Defining the meaning of content in an argument |
| Evaluation | Examining content | Evaluating the credibility of sources and claims |
| Inference | Inferring conclusions | Drawing conclusions based on evidence, observation, experience, and values |
| Explanation | Inferring conclusions | Justifying one's own judgments by providing evidence and assumptions |
| Induction | Inferring conclusions | Estimating the results that can be drawn based on past experience and facts |
| Deduction | Inferring conclusions | Judging what to do based on already-known facts |
| Numeracy | Overall CT skills | Making decisions based on quantitative information such as figures and graphs |
Lastly, The WGCTA (Waston & Glaser, 2002) was not developed based on the specific construct already proposed but focused on the logical aspects of CT skills. This CT test comprises the following five subscales: inference, recognition of assumptions, deduction, interpretation, and evaluation of argument. Table 6 summarizes the characteristics of the WGCTA. As a result of exploratory factor analysis, it was confirmed the uni-dimensionality of this CT test (Hassan & Madhum, 2007). This result supports the idea that the test primarily measures the logical aspects of CT skills. In addition, the result of the overall test reliability revealed that the reliability coefficient (Cronbach’s alpha) was .79, which confirms sufficient internal consistency of the CT tests compared with other CT tests.
| Subscale | Related CT Skills | Item Characteristics |
|---|---|---|
| Inference | Inferring conclusions | Judging the extent to which proposed inferences can be drawn from given statements |
| Recognition of assumptions | Inferring conclusions | Identifying assumptions of opinions that can be drawn from the given statements |
| Deduction | Inferring conclusions | Judging whether the proposed conclusion can be drawn from the given premises |
| Interpretation | Inferring conclusions | Judging whether the proposed conclusions can be logically drawn from the given paragraphs |
| Evaluation of argument | Examining content | Judging whether the basis for the conclusion in an argument is valid |
As shown in Tables 4, 5, and 6, a variety of test items were developed to measure the CT skills of test-takers. In addition, the developed CT tests tended to have fewer test items that corresponded to the CT skills of identifying the main point and examining content in general. This is probably because these tests were developed for native English speakers, who were assumed to have sufficient skills for understanding and examining the content. However, EFL learners are less proficient than native English speakers. Thus, when CT tests and CT assessment models are developed for EFL learners, it is necessary to provide test items and criteria that focus on the skills of analyzing and examining content correctly.
Conversely, many test items corresponded to CT skills of inferring conclusions. This tendency seems to be because drawing correct conclusions is the final step of CT skills and requires higher-order thinking skills. This finding is also consistent with the characteristics of questions related to CT skills in Japanese English textbooks (Hirai & Kubo, 2023). In other words, questions that require students to form opinions based on evidence, which corresponds to the CT skills of inferring conclusions, were also prevalent in Japanese elementary, junior high, and high school English textbooks. However, because the skills of comprehending content accurately and examining information are essential to infer correct conclusions (Ennis, 2015; Kusumi & Michita, 2015), these skills are also necessary for developing CT skills. Therefore, more test items and questions related to CT skills of identifying the main point and examining content must be provided for Japanese EFL learners, contrary to the trend in previous studies. Thus, CT skills should be developed progressively and appropriately according to the level of Japanese EFL learners.
Although the constructs of all CT tests are not entirely consistent, Table 7 summarizes their characteristics by surveying the similarities among previous studies, as mentioned above. Similar to the other Tables, the names of CT skills already established in this study were adopted in Table 7. Notably, the test items that focused on the skill of metacognizing one’s own thoughts could not be identified in the above CT tests. Because this skills are related to CT in general (Kusumi, 2018), it would be difficult to establish the subcomponent to measure metacognitive skills. However, metacognition affects not only the cognitive aspects but also the affective aspects, and is essential in the problem-solving and decision-making process (Ennis, 1987). While it is difficult to measure metacognition through test items or tasks, it seems necessary to present it to learners as a disposition. Thus, the key dispositions for metacognition are presented as characteristics in Table 7. CT skills could be validly measured and evaluated by creating tasks, test items, and rublic based on the CT characteristics as shown in Table 7.
| Construct of CT Skills | Characteristics |
|---|---|
| Identifying main points | ・Identifying main points such as claims from the content ・Identifying the meaning of ambiguous terms ・Understanding the content of statistical material |
| Examining content | ・Choosing credible information from the given information ・Judging the validity of the connection between the evidence and the claims of the given information |
| Inferring conclusions | ・Judging whether conclusions can be drawn from the content ・Choosing an appropriate method to examine the presented hypotheses and results ・Inferring conclusions that can be drawn based on facts |
| Metacognizing one’s own thoughts | ・Attempting to reflect on their own cognitive processes |
This study investigated the constructs of CT skills proposed in previous studies on the theoretical background of CT and CT tests. Based on the results of this review, recommendations were made for the assessment and cultivation of CT skills in the future EFL content. Regarding the theoretical background of CT, it was revealed that the construct of CT skills mainly comprises: (a) identifying main points, (b) examining content, (c) inferring conclusions, and (d) metacognizing one’s own thoughts. From the perspective of CT tests, this study identified the characteristics of test items for measuring the construct of CT skills (see Table 7). Future research should develop materials such as tasks, rubrics, and tests for cultivating and evaluating CT skills based on the results of the current study.
A limitation of previous studies is that their constructs of CT skills differed slightly (e.g., Sato, 2022). Furthermore, while multiple-choice CT tests for EFL learners have been validated, CT assessment models have not been established because these CT tests have not been able to comprehensively validate the CT construct. The current study is useful in addressing these gaps. Therefore, it is expected that assessment rubrics and test items for evaluating CT skills should be developed with reference to the CT constructs identified in the present study. The results of this review could also contribute to designing course plans and creating materials to develop CT skills.
However, the CT studies reviewed in this study were designed for native English speakers. Therefore, it is assumed that some CT skills require much higher-order thinking skills for EFL learners. For example, CT skills of inference presented by Ennis (1987) sometimes require rhetorical skills, such as syllogism. The test items corresponding to inference in Hirai et al. (2020) were also calculated with high item difficulty (item difficulty = .39). In addition, Japanese English textbooks and practical studies that aim to develop CT skills tend to encourage the formation of opinions, which corresponds to CT skills of inferring conclusions (e.g., Imai & Mineshima, 2017; Hirai & Kubo, 2023). Nonetheless, such production-based CT skills are expected to be a difficult task for many EFL learners because they require higher-order thinking skills. Therefore, it would be desirable to also develop test items and rubric criteria that focus on the reception-based CT skills of identifying the main point and examining content. Varaporn and Sitthitikul (2019) also mention that it is important to develop skills such as understanding the outline and main points of information and examining the consistency of the logical process, which corresponds to the receptive-based CT skills mentioned above. Consequently, materials that focus more on the CT skills EFL learners need to cultivate must be created to support future learners.
