In this rejoinder, we respond to two commentaries on the study by Greiff, S.; Stadler, M.; Sonnleitner, P.; Wolff, C.; Martin, R. Sometimes less is more: Comparing the validity of complex problem solving measures. Intelligence 2015, 50, 100–113. The study was the first to address the important comparison between a classical measure of complex problem solving (CPS) and the more recent multiple complex systems (MCS) approach regarding their validity. In the study, we investigated the relations between one classical microworld as (...) the initially developed method (here, the Tailorshop) with three more recently developed multiple complex systems (MCS; here, MicroDYN, Genetics Lab, and MicroFIN) tests. We found that the MCS tests showed higher levels of convergent validity with each other than with the Tailorshop even after reasoning was controlled for, thus empirically distinguishing between the two approaches. The commentary by Kretzschmar and the commentary by Funke, Fischer, and Holt expressed several concerns with how our study was conducted, our data was analyzed, and our results were interpreted. Whereas we acknowledge and agree with some of the more general statements made in these commentaries, we respectfully disagree with others, or we consider them to be at least partially in contrast with the existing literature and the currently available empirical evidence. (shrink)

Influencing students’ educational achievements first requires understanding the underlying processes that lead to variation in students’ performance. Researchers are therefore increasingly interested in analyzing the differences in behavior displayed in educational assessments rather than merely assessing their outcomes. Such analyses provide valuable information on the differences between successful and unsuccessful students and help to design appropriate interventions. Complex problem solving (CPS) tasks have proven to provide particularly rich process data as they allow for a multitude of behaviors several of which (...) can lead to a successful performance. So far, this data has often been analyzed on a rather aggregated level looking at an average number of actions or predefined strategies. with only a few articles investigating the specific actions performed. In this paper, we illustrate and discuss a more detailed report the results of an exploratory analysis of CPS process data based on the specific actions performed.log-files that is aimed at distinguishing between students that applied the correct strategy to a problem but failed to solve it and those applying the strategy successfully. In that, the sequence of behavior displayed is reduced to interpretable parts (n-grams) that allow searching for meaningful differences between the two groups of students. This level of analysis allows finding previously undefined or unknown patterns within the data and increases our understanding of the processes underlying successful problem -solving behavior even further. (shrink)

In this paper we propose the multiple complex systems approach for assessing domain-general complex problem-solving skills and its processes knowledge acquisition and knowledge application. After defining the construct and the formal frameworks for describing complex problems, we emphasise some of the measurement issues inherent in assessing CPS skills with single tasks. With examples of the MicroDYN test and the MicroFIN test, we show how to adequately score problem-solving skills by using multiple tasks. We discuss implications for problem-solving research and the (...) assessment of CPS skills in general. (shrink)

Open access to information is now a universal phenomenon thanks to rapid technological developments across the globe. This open and universal access to information is a key value of democratic societies because, in principle, it supports well-informed decision-making on individual, local, and global matters. In practice, however, without appropriate readiness for navigation in a dynamic information landscape, such access to information can become a threat to public health, safety, and economy, as the COVID-19 pandemic has shown. In the past, this (...) readiness was often conceptualized in terms of adequate literacy levels, but the contemporarily observed highest-ever literacy levels have not immunized our societies against the risks of misinformation. Therefore, in this Perspective, we argue that democratization of access to information endows citizens with new responsibilities, and second, these responsibilities demand readiness that cannot be reduced to mere literacy levels. In fact, this readiness builds on individual adequate literacy skills, but also requires rational thinking and awareness of own information processing. We gather evidence from developmental, educational, and cognitive psychology to show how these aspects of readiness could be improved through education interventions, and how they may be related to healthy work-home balance and self-efficacy. All these components of education are critical to responsible global citizenship and will determine the future direction of our societies. (shrink)

This paper summarizes a theory of cognitive development and elaborates on its educational implications. The theory postulates that development occurs in cycles along multiple fronts. Cognitive competence in each cycle comprises a different profile of executive, inferential, and awareness processes, reflecting changes in developmental priorities in each cycle. Changes reflect varying needs in representing, understanding, and interacting with the world. Interaction control dominates episodic representation in infancy; attention control and perceptual awareness dominate in realistic representations in preschool; inferential control and (...) awareness dominate rule-based representation in primary school; truth and validity control and precise self-evaluation dominate in principle-based thought in adolescence. We demonstrate that the best predictors of school learning in each cycle are the cycle’s cognitive priorities. Also learning in different domains, e.g., language and mathematics, depends on an interaction between the general cognitive processes dominating in each cycle and the state of the representational systems associated with each domain. When a representational system is deficient, specific learning difficulties may emerge, e.g., dyslexia and dyscalculia. We also discuss the educational implications for evaluation and learning at school. (shrink)

Psychometrically sound short scales are required to comprehensively and yet economically assess fundamental motives in research settings such as large-scale assessments. In order to provide such a time- and cost-efficient instrument, we conducted three studies to develop further and validate 16 German scales with three items each assessing fundamental motives [16 motives research scales ]. In Study 1, we applied a top–down construction process to develop a preliminary item pool on the basis of a thorough revision of existing construct definitions. (...) In Study 2, we chose an approach that allowed us to balance the optimization of psychometric properties with content coverage to select three-item scales for each of the 16 motives. For the item selection process, we combined exploratory factor analyses, confirmatory factor analyses, ant colony optimization algorithm, and Mokken scale analyses. In Study 3, we cross-validated the results obtained in Study 2 and placed the 16mrs in a nomological network consisting of Big Five traits and Power, Achievement, Affiliation, Intimacy, and Fear motives. The results of these studies indicate that the 16mrs can be used to reliably and validly assess fundamental motives that represent a level of personality that differs from the Big Five and covers motivational aspects beyond the well-established Power, Achievement, Affiliation, Intimacy, and Fear motives. Limitations concerning the reliability of the Autonomy scale and the empirical discrimination of the Dominance and Status scales are discussed. In addition to the validated German version, we also provide the English translation of the items, which, however, need to be validated before use. (shrink)