The CAT is entirely dedicated to assessing the critical-thinking skills involved in scientific reasoning and practical problem solving. While the test is found to have reasonable content validity, various issues with its prompts are discussed, along with significant issues with its scoring. The CAT’s recommended use as a “model” for curricular changes, called CAT Apps, is criticized as “teaching to the test.”.

The Collegiate Learning Assessment Test has become popular and highly recommended, praised for its reliability and validity. I argue that while the CLA may be a commendable test for measuring critical-thinking, problem-solving, and logical-reasoning skills, those who are scoring students’ answers to the test’s questions are rendering the CLA invalid.

In spite of significant research efforts, argument technologies do not seem poised to scale up as much as most commentators would hope or even predict. In this paper, I discuss what obstacles bar the way to more widespread success of argument technologies and venture some suggestions on how to circumvent such difficulties: doing so will require a significant shift in how this research area is typically understood and practiced. I begin by exploring a much broader yet closely related question: To (...) what extent are people natively good at arguing? This issue has always been central to philosophical reflection and it has become even more urgent nowadays, with the explosion of persuasive technologies and unprecedented opportunities for large-scale social influence. The answer hinges on what aspect of argumentation is taken under consideration: evidence suggests that people are relatively bad at analyzing the structure of arguments, especially when these are presented out of context and in abstract terms; in contrast, data show that even laymen tend to excel in the interactive practice of argumentation, in particular when motivation is high and something significant is at stake. Unfortunately, current argument technologies are more closely tailored to the former type of activity than to the latter, which is the main reason behind their relative lack of success with the general public. Changing this state of affair will require a commitment to ecological argument technologies: that is, technologies designed to support real-time, engaging and meaningful argumentative interactions performed by laypeople in their ordinary life, instead of catering to the highly specific needs of a minority of niche users. (shrink)

Multiple-choice questions have an undeserved reputation for only being able to test student recall of basic facts. In fact, well-crafted mechanically gradable questions can measure very sophisticated cognitive skills, including those engaged at the highest level of Benjamin Bloom’s taxonomy of outcomes. In this article, I argue that multiple-choice questions should be a part of the diversified assessment portfolio for most philosophy courses. I present three arguments broadly related to fairness. First, multiple-choice questions allow one to consolidate subjective decision making (...) in a way that makes it easier to manage. Second, multiple-choice questions contribute to the diversity of an evaluation portfolio by balancing out problems with writing-based assessments. Third, by increasing the diversity of evaluations, multiple-choice questions increase the inclusiveness of the course. In the course of this argument I provide examples of multiple-choice questions that measure sophisticated learning and advice for how to write good multiple-choice questions. (shrink)

This essay offers a comprehensive vision for a higher education program incorporating critical thinking across the curriculum at hypothetical Alpha College, employing a rigorous detailed conception of critical thinking called “The Alpha Conception of Critical Thinking”. The program starts with a 1-year, required, freshman course, two-thirds of which focuses on a set of general critical thinking dispositions and abilities. The final third uses subject-matter issues to reinforce general critical thinking dispositions and abilities, teach samples of subject matter, and introduce subject-specific (...) critical thinking. Subject-matter departmental and other units will make long-range plans for incorporating critical thinking in varying amounts in subject-matter courses, culminating in a written Senior Thesis/Project involving investigating, taking, and defending a position, which reinforce critical thinking abilities and dispositions and increase subject-matter knowledge. Teaching approaches used in the program are involving and based on the principle, “We learn what we use.” Both summative and formative assessment are employed as appropriate. Coordination and support are extensive. Objections and concerns are discussed, and alternatives, including possible transitions, are considered. An extended review of research supports moving toward CTAC. (shrink)

A large body of evidence suggests that the Personalized System of Instruction (PSI) improves learning. In courses that use PSI, the material is divided into units, students must pass a test on each unit before advancing to the next unit, there’s no group-level instruction, and students advance in the course at their own pace. While studies find that PSI improves learning outcomes in a wide range of settings, researchers haven’t studied the effectiveness of PSI in critical thinking classes. In this (...) paper, I argue that teaching critical thinking with PSI can effectively promote critical thinking skills. I describe a course that uses PSI to teach critical thinking and I present evidence from pre- and post-tests that indicates that students substantially improved their reasoning skills in this course. I also discuss the costs and drawbacks of using this method of instruction and consider ways of addressing these problems. (shrink)

In summaries of “best practices” for pedagogy, one typically encounters enthusiastic advocacy for metacognition. Some researchers assert that the body of evidence supplied by decades of education studies indicates a clear pedagogical imperative: that if one wants their students to learn well, one must implement teaching practices that cultivate students’ metacognitive skills. -/- In this dissertation, I counter that education research does not impose such a mandate upon instructors. We lack sufficient and reliable evidence from studies that use the appropriate (...) research design to validate the efficacy of metacognitive skill-building interventions (not just evaluate their relationship to learning outcomes). I argue that improved academic outcomes following these interventions aren’t necessarily mediated by increased metacognitive skills; rather, enhanced student performance can be accounted for by other factors that accompany metacognitive training, particularly the explicit provision of domain-specific knowledge. -/- On the way to this conclusion, I elaborate some complications and controversies surrounding “metacognition.” This is a sprawling and nebulous construct, which makes generalizations about its pedagogical value dubious from the outset. Moreover, it is unclear the extent to which the end goal of metacognitive skill-building (cognitive self-mastery, involving knowledge of and control over our own minds) is even possible, given our mental architecture; some cognitive scientists allege that we are subject to phenomenological illusions which make this seem more achievable than it actually is. -/- I also attempt to provide an account of how metacognitive skill-building could receive glowing endorsements from educators and education researchers, even though its conceptual and empirical underpinnings are flimsy. The error theory I offer identifies two major factors that may foster belief in the efficacy of metacognitive training: goalpost-shifting around the objective of such efforts, and motivated reasoning in defense of the desirable conclusion that educators can significantly reshape students’ minds and unlock their intellectual potential through simple pedagogical interventions. (shrink)