This chapter contains sections titled: Measuring Learning for Accountability Purposes The Theory of Assessment Assessment and Fairness Conclusion.

Consciousness can be measured in various ways, but different measures often yield different conclusions about the extent to which awareness relates to performance. Here, we compare five different subjective measures of awareness in the context of an artificial grammar learning task. Participants expressed their subjective awareness of rules using one of five different scales: confidence ratings , post-decision wagering , feeling of warmth , rule awareness , and continuous scale . All scales were equally sensitive to conscious knowledge. PDW, (...) however, was affected by risk aversion, and both RAS and SDS applied different minimal criteria for rule awareness. CR seems to capture the largest range of consciousness, but failed to indicate unconscious knowledge with the guessing criterion. We close by discussing the theoretical implications of scale sensitivity and propose that CR’s unique features enable a finer assessment of subjective states of awareness. (shrink)

This study takes a holistic view of flow and anti-flow experiences as interactive subsystems in blended English as a foreign language learning and examines the dynamic complex construct in the field of instructed second language acquisition. We first rephrased the 22-item Classroom Flow Questionnaire to better reflect the context of blended EFL learning. The modified CFQ was then administered to 661 first language Chinese EFL learners. A final 14-item Foreign Language Flow Scale was developed based on results from (...) a series of reliability and validity tests. Both exploratory and confirmatory factor analysis results have demonstrated that foreign language learning flow is a three-dimensional construct involving Enjoyment, Boredom, and Anxiety, thus conceptualizing and validating flow as a continuum with both positive and negative ends. Moreover, participants reported that they experienced the lowest degree of enjoyment, while with respect to the negative flow, they almost experienced similar degree of boredom and anxiety. The present study contributes to the development of the conceptual framework for flow in ISLA as well as constructive pedagogical implications for L2 researchers and educators. Suggestions for future research are also provided. (shrink)

Studies of implicit learning have shown that individuals exposed to a rule-governed environment often learn to exploit 'rules' which describe the structural relationship between environmental events. While some authors have interpreted such demonstrations as evidence for functionally separate implicit learning systems, others have argued that the observed changes in performance result from explicit knowledge which has been inadequately assessed. In this paper we illustrate this issue by considering one commonly used implicit learning task, the Serial reaction time (...) task, and outline what we see as an important problem associated with each of the commonly used methods used to assess explicit knowledge. This is that each measure requires a form of response which is dependent on the subjects having some knowledge of the serial-order of the sequence. We argue that such methods, or more specifically their analyses, seriously underestimate other sources of knowledge, which may be available to subjects during their performance of the SRT task. In support of this argument we demonstrate that subjects' serial-order knowledge can, in principle, be independent of subjects' knowledge of the statistical structure of the sequence, and we propose an alternative method for analysing performance on the Generate task which avoids this problem. (shrink)

Comparing the relative sensitivity of direct and indirect measures of learning is proposed as the best way to provide evidence for unconscious learning when both conceptual and operative definitions of awareness are lacking. This approach was first proposed by Reingold & Merikle (1988) in the context of subliminal perception. In this paper, we apply it to a choice reaction time task in which the material is generated based on a probabilistic finite-state grammar (Cleeremans, 1993). We show (1) that (...) participants progressively learn about the statistical structure of the stimulus material over training with the choice reaction time task, and (2) that they can use some of this knowledge to predict the location of the next stimulus in a subsequent “generation” task. However, detailed partial correlational analyses of the correspondence between performance during the reaction time task and the statistical structure of the training material showed that large effects remained even when controlling for explicit knowledge as assessed by the generation task. Hence we conclude (1) that at least some of the knowledge expressed through reaction time performance can not be characterized as conscious, and (2) that even when associations are found at a global level of analysis, dissociations can still be obtained when more detailed analyses are conducted. Finally, we also show that participants are limited in the depth of the contingencies they can learn about, and that these limitations are shared by the Simple Recurrent Network model of Cleeremans & McClelland (1991). (shrink)

Comparing the relative sensitivity of direct and indirect measures of learning is proposed as the best way to provide evidence for unconscious learning when both conceptual and operative definitions of awareness are lacking. This approach was first proposed by Reingold & Merikle (1988) in the context of subliminal perception. In this paper, we apply it to a choice reaction time task in which the material is generated based on a probabilistic finite-state grammar (Cleeremans, 1993). We show (1) that (...) participants progressively learn about the statistical structure of the stimulus material over training with the choice reaction time task, and (2) that they can use some of this knowledge to predict the location of the next stimulus in a subsequent “generation” task. However, detailed partial correlational analyses of the correspondence between performance during the reaction time task and the statistical structure of the training material showed that large effects remained even when controlling for explicit knowledge as assessed by the generation task. Hence we conclude (1) that at least some of the knowledge expressed through reaction time performance can not be characterized as conscious, and (2) that even when associations are found at a global level of analysis, dissociations can still be obtained when more detailed analyses are conducted. Finally, we also show that participants are limited in the depth of the contingencies they can learn about, and that these limitations are shared by the Simple Recurrent Network model of Cleeremans & McClelland (1991). (shrink)

Agnotology is the study of how ignorance arises via circulation of misinformation calculated to mislead. Legates et al. had questioned the applicability of agnotology to politically-charged debates. In their reply, Bedford and Cook, seeking to apply agnotology to climate science, asserted that fossil-fuel interests had promoted doubt about a climate consensus. Their definition of climate ‘misinformation’ was contingent upon the post-modernist assumptions that scientific truth is discernible by measuring a consensus among experts, and that a near unanimous consensus exists. However, (...) inspection of a claim by Cook et al. of 97.1 % consensus, heavily relied upon by Bedford and Cook, shows just 0.3 % endorsement of the standard definition of consensus: that most warming since 1950 is anthropogenic. Agnotology, then, is a two-edged sword since either side in a debate may claim that general ignorance arises from misinformation allegedly circulated by the other. Significant questions about anthropogenic influences on climate remain. Therefore, Legates et al. appropriately asserted that partisan presentations of controversies stifle debate and have no place in education. (shrink)

A citizens’ movement in Berlin advocates for the expropriation of housing corporations and has won a significant majority in a popular referendum in September 2021. Building on this proposal, this paper develops a general account of expropriation as a measure for corporate reform and thereby contributes to the ongoing debate on the democratic accountability of business corporations. It argues that expropriation is a valuable tool for intervention in a dire situation in some economic sector to enable a re-structuring of the (...) governance of the assets in question. Compared with other tools available, expropriation is a more forward-looking, genuinely political measure that does not depend on the legal assignment of guilt but rather proceeds in a pragmatic and problem-oriented manner. It also allows us to reconsider in how far the market mechanism should be employed in the administration of assets. Objections from private property rights against expropriation fail as corporations generally are privileged, quasi-public institutions that can justifiably be subject to democratic interventions. Expropriation is thus an important addition to the arsenal of corporate reform proposals, especially for those concerned with a broad democratization of the corporation. (shrink)

As machine learning informs increasingly consequential decisions, different metrics have been proposed for measuring algorithmic bias or unfairness. Two popular “fairness measures” are calibration and equality of false positive rate. Each measure seems intuitively important, but notably, it is usually impossible to satisfy both measures. For this reason, a large literature in machine learning speaks of a “fairness tradeoff” between these two measures. This framing assumes that both measures are, in fact, capturing something important. To date, philosophers have (...) seldom examined this crucial assumption, and examined to what extent each measure actually tracks a normatively important property. This makes this inevitable statistical conflict – between calibration and false positive rate equality – an important topic for ethics. In this paper, I give an ethical framework for thinking about these measures and argue that, contrary to initial appearances, false positive rate equality is in fact morally irrelevant and does not measure fairness. (shrink)

This study applied cognitive diagnostic models to assess students’ learning progressions in energy. A Q-matrix was proposed based on existing literature about learning progressions of energy in the physical science domain and the Trends in International Mathematics and Science Study assessment framework. The Q-matrix was validated by expert review and real data analysis. Then, the deterministic inputs, noisy ‘and’ gate model with hierarchical relations was applied to data from three jurisdictions that had stable, defined science curricula. The results (...) suggested that the hypothesized learning progression was consistent with the observed progression in understanding the energy concept. We also found similarities in students’ attribute mastery across the three jurisdictions. In addition, we examined the instructional sensitivity of the selected item. We discuss several curriculum-related issues and student misconceptions that may affect students’ learning progressions and mastery patterns in different regions of the world. (shrink)

This paper considers different subjective measures of conscious and unconscious knowledge in a concept formation paradigm. In particular, free verbal reports are compared with two subjective measures, the zero-correlation and the guessing criteria, based on trial-by-trial confidence ratings (a type of on-line verbal report). Despite the fact that free verbal reports are frequently dismissed as being insensitive measures of conscious knowledge, a considerable bulk of research on implicit learning has traditionally relied on this measure of consciousness, because it is (...) widely regarded as almost self-evident that the content of any conscious state that is intentional and conceptual can be expressed verbally. However, we found that the most recently developed subjective measures based on trial-by-trial confidence ratings provided a more sensitive measure of conscious and unconscious knowledge than free verbal reports. In a complementary way, the qualitative pattern of the free report and the confidence measures were similar, providing further evidence for the validity of the latter. (shrink)

We investigate many paradigms of identifications for classes of languages in a measure-theoretic context, and we relate such paradigms to their analogues in learning on informants. Roughly speaking, the results say that most paradigms in measure-theoretic learning wrt some classes of distributions are equivalent to the corresponding paradigms for identification on informants.

Overgaard, Timmermans, Sandberg, and Cleeremans ask if the conscious experience of people in implicit learning experiments can be explored more fully than just confidence ratings allow. We show that confidence ratings play a vital role in such experiments, but are indeed incomplete in themselves: in addition, use of structural knowledge attributions and ratings of fringe feelings like familiarity are important in characterizing the phenomenology of the application of implicit knowledge.

From a theoretical perspective, most discussions of statistical learning have focused on the possible “statistical” properties that are the object of learning. Much less attention has been given to defining what “learning” is in the context of “statistical learning.” One major difficulty is that SL research has been monitoring participants’ performance in laboratory settings with a strikingly narrow set of tasks, where learning is typically assessed offline, through a set of two-alternative-forced-choice questions, which follow a (...) brief visual or auditory familiarization stream. Is that all there is to characterizing SL abilities? Here we adopt a novel perspective for investigating the processing of regularities in the visual modality. By tracking online performance in a self-paced SL paradigm, we focus on the trajectory of learning. In a set of three experiments we show that this paradigm provides a reliable and valid signature of SL performance, and it offers important insights for understanding how statistical regularities are perceived and assimilated in the visual modality. This demonstrates the promise of integrating different operational measures to our theory of SL. (shrink)

Edward Lee Thorndike (August 31, 1874 Williamsburg, Massachusetts 1] - August 9, 1949) was an American psychologist who spent nearly his entire career at Teachers College, Columbia University. His work on animal behavior and the learning process led to the theory of connectionism and helped lay the scientific foundation for modern educational psychology. He also worked on solving industrial problems, such as employee exams and testing. He was a member of the board of the Psychological Corporation, and served as (...) president of the American Psychological Association in 1912 This book, Measurements Of Twins (1905), by Edward L. Thorndike, is a replication of a book originally published before 1905. It has been restored by human beings, page by page, so that you may enjoy it in a form as close to the original as possible. This book was created using print-on-demand technology. Thank you for supporting classic literature. (shrink)

Dienes and Seth (2010) conclude that confidence ratings and post-decision wagering are two comparable and recommendable measures of conscious experience. In a recently submitted paper, we have however found that both methods are problematic and seem less suited to measure consciousness than a direct introspective measure. Here, we discuss the methodology and conclusions put forward by Dienes and Seth, and why we think the two experiments end up with so different recommendations.

Previous research has shown that emotionally-valenced words are given higher judgements of learning (JOLs) than are neutral words. The current study examined potential explanations for this emotional salience effect on JOLs. Experiment 1 replicated the basic emotionality/JOL effect. In Experiments 2A and 2B, we used pre-study JOLs and assessed memory beliefs qualitatively, finding that, on average, participants believed that positive and negative words were more memorable than neutral words. Experiment 3 utilised a lexical decision task, resulting in lower reaction (...) times (RTs) for positive words than for neutral words, but equivalent RTs for negative and neutral words, suggesting that processing fluency may partially account for higher JOLs for positive words, but not for negative words. Finally, we conducted a series of moderation analyses in Experiment 4 which assessed the relative contributions of fluency and beliefs to JOLs by measuring both factors in the same participants, showing that RTs made no significant contribution to JOLs for either positive or negative words. Our findings suggest that although positive words may be more fluently processed than neutral words, memory beliefs are the primary factor underlying higher JOLs for both positive and negative words. (shrink)

This study is framed under the student approaches to learning tradition. The aim was to identify convergence in quantitative and qualitative responses of individuals when measuring their conceptions of and approaches to learning with a mixed methods design. A sample of 1110 Spanish Master’s level teacher education students completed a scale on approaches to learning, and a randomly selected subsample of 111 answered an open-ended question on how they learned. Overall, the qualitative and quantitative data did not (...) support each other, as inventory responses showed a clear predominance of a deep, non-surface approach to learning, whereas qualitative answers reflected a tendency towards lower-order conceptions of learning. Inconsistencies in the results suggest future research ought to use a combination of techniques when exploring constructs such as learning should they wish to draw valid conclusions. (shrink)

Self-regulation of learning is a key psychological factor that supports young athletes aiming to reach the elite level by promoting their involvement in deliberate practice. We contributed to the validation of the Italian version of the Bartulovic et al. Self-Regulation of Learning – Self-Report Scale for Sport Practice by testing its factorial structure, reliability, and measurement invariance among elite and non-elite football players, involving 415 male professional, semi-professional, and amateur youth academy players. The original six-factor structure did (...) not fit the data well and a five-factor solution was a better fit. This five-factor solution was measurement invariant across groups of elite and non-elite athletes. We found that elite athletes scored significantly higher than non-elite ones in each SRL subprocess. Implications for future validation studies and for the use of this tool are discussed. (shrink)

Foundations of Bayesianism is an authoritative collection of papers addressing the key challenges that face the Bayesian interpretation of probability today. Some of these papers seek to clarify the relationships between Bayesian, causal and logical reasoning. Others consider the application of Bayesianism to artificial intelligence, decision theory, statistics and the philosophy of science and mathematics. The volume includes important criticisms of Bayesian reasoning and also gives an insight into some of the points of disagreement amongst advocates of the Bayesian approach. (...) The upshot is a plethora of new problems and directions for Bayesians to pursue.The book will be of interest to graduate students or researchers who wish to learn more about Bayesianism than can be provided by introductory textbooks to the subject. Those involved with the applications of Bayesian reasoning will find essential discussion on the validity of Bayesianism and its limits, while philosophers and others interested in pure reasoning will find new ideas on normativity and the logic of belief. (shrink)

Supervised machine learning has found its way into ever more areas of scientific inquiry, where the outcomes of supervised machine learning applications are almost universally classified as predictions. I argue that what researchers often present as a mere terminological particularity of the field involves the consequential transformation of tasks as diverse as classification, measurement, or image segmentation into prediction problems. Focusing on the case of machine-learning enabled poverty prediction, I explore how reframing a measurement problem (...) as a prediction task alters the primary epistemic aim of the application. Instead of measuring a property, machine learning developers conceive of their models as predicting a given measurement of this property. I argue that this _predictive reframing_ common to supervised machine learning applications is epistemically and ethically problematic, as it allows developers to externalize concerns critical to the epistemic validity and ethical implications of their model’s inferences. I further hold that the predictive reframing is not a necessary feature of supervised machine learning by offering an alternative conception of machine learning models as measurement models. An interpretation of supervised machine learning applications to measurement tasks as _automatically-calibrated model-based measurements_ internalizes questions of construct validity and ethical desirability critical to the measurement problem these applications are intended to and presented as solving. Thereby, this paper introduces an initial framework for exploring technical, historical, and philosophical research at the intersection of measurement and machine learning. (shrink)

The distinction between students diagnosed with a learning disability and those considered merely slow learners is based on conceptually flawed assumptions that: 1) LD represents a brain dysfunction while SL does not; 2) LD is a well-defined disorder; 3) valid measurement instruments distinguish LD and SL; 4) special education for students with LD is fundamentally different from that for SL students. These erroneous beliefs are maintained because governmental legislation transformed a diagnosis of LD into an admission ticket to (...) a variety of government entitlements thereby subjecting it to political, legal, social and ideological forces. (shrink)

The world today is characterized by intercultural diversity. More and more communication takes place between people with different linguistic as well as cultural backgrounds. This happens because of contacts within the areas of business, science, education etc. but also because of immigration brought about by labour shortage or unstable political situation. The globalisation of the economy with increased appreciation by companies that managing cultural differences properly can be a key factor in getting things done effectively across borders. With increased contact (...) of personnel and customers from diverse cultural background, there is a growing demand for businesses to understand and manage the diverse values, perceptions, business worldviews and behaviour of corporations, staff, and its customer. Intercultural communication and management is an interdisciplinary human resources field concerned with faciliting communication, management and effective interaction of personnel and customers across borders. The starting point for understanding consumer behaviour is really important. Marketing and environmental stimuls enter the consumer’s conscioussnes, and set of psychological processes combine with certain consumer characteristics to resault in decision processes and purchase decision. The marketer’s task is to understand what happened in the consumer’s consciousness between the arrival of the outside marketing stimuli and the ultimate purchasedecision. Four psychological processes are motivation, perception, learning and also memroy, fundamentaly influence consumer responses. Understanding consumer behaviour may often be complicated since many different factors influence the behaviour and sicnce many different forms of behaviour exist. Paper describe fuzzy concept for evaluation of some cros-cultural impact and compare it with other type of measurements presented in literature. (shrink)

In the recent literature on confirmation there are two leading approaches to the provision of a probabilistic measure of the degree to which a hypothesis is confirmed by evidence. The first is to construe the degree to which evidence E confirms hypothesis H as a function that is directly proportional to p and inversely proportional to p . I shall refer to this as the probability approach. The second approach construes the notion as a function that is directly proportional to (...) the true-positive rate – the probability of the evidence if the hypothesis is true, p – and inversely proportional to the false-positive rate – the probability of the evidence if the hypothesis is false, p . These reverse conditional probabilities – of the evidence on the truth or falsehood of the hypothesis – are sometimes known as likelihoods. I shall refer to the approach to confirmation that uses them as the likelihood approach.For each of these approaches, there are two obvious options as to how to define the confirmation function. The first is to define it as the difference between the magnitude that is treated as increasing confirmation or p ) and the magnitude that is treated as decreasing confirmation or p ). The second is to define confirmation as the quotient of these two magnitudes.This yields four different measures of confirmation, represented in the following table: Differences RatiosProbabilities PD = p – p1 PR = Graphic2Likelihoods LD = p – p3 LR = Graphic4All these measures agree on whether or not E provides some support for H. Intuitively, we want to say that E provides some support for H just in case learning that E is true would make us assign a higher probability to H, i.e. when p(H …. (shrink)

Clustering plays an important role in data mining, pattern recognition, and machine learning. Single-valued neutrosophic sets are useful means to describe and handle indeterminate and inconsistent information that fuzzy sets and intuitionistic fuzzy sets cannot describe and deal with. To cluster the data represented by single-valued neutrosophic information, this article proposes single-valued neutrosophic clustering methods based on similarity measures between SVNSs. First, we define a generalized distance measure between SVNSs and propose two distance-based similarity measures of SVNSs. Then, we (...) present a clustering algorithm based on the similarity measures of SVNSs to cluster single-valued neutrosophic data. Finally, an illustrative example is given to demonstrate the application and effectiveness of the developed clustering methods. (shrink)

In previous work, we have combined computable structure theory and algorithmic learning theory to study which families of algebraic structures are learnable in the limit (up to isomorphism). In this paper, we measure the computational power that is needed to learn finite families of structures. In particular, we prove that, if a family of structures is both finite and learnable, then any oracle which computes the Halting set is able to achieve such a learning. On the other hand, (...) we construct a pair of structures which is learnable but no computable learner can learn it. (shrink)