Process reliabilist accounts claim that a belief is justified when it is the result of a reliable belief-forming process. Yet over what range of possible token processes is this reliability calculated? I argue against the idea that _all_ possible token processes (in the actual world, or some other subset of possible worlds) are to be considered using the case of a user acquiring beliefs based on the output of an AI system, which is typically reliable for a substantial local (...) range but unreliable when all possible inputs are considered. I show that existing solutions to the generality problem imply that these cases cannot be solved by a more fine-grained typing of the belief-forming process. Instead, I suggest that reliability is evaluated over a range restricted by the content of the actual belief and by the similarity of the input to the actual input. (shrink)

A measure of coherence is said to be truth conducive if and only if a higher degree of coherence (as measured) results in a higher likelihood of truth. Recent impossibility results strongly indicate that there are no (non-trivial) probabilistic coherence measures that are truth conducive. Indeed, this holds even if truth conduciveness is understood in a weak ceteris paribus sense (Bovens & Hartmann, 2003, Bayesian epistemology. New York, Oxford: Oxford University Press; Olsson, 2005, Against coherence: Truth probability and justification. Oxford: (...) Oxford University Press). This raises the problem of how coherence could nonetheless be an epistemically important property. Our proposal is that coherence may be linked in a certain way to reliability. We define a measure of coherence to be reliability conducive if and only if a higher degree of coherence (as measured) results in a higher probability that the information sources are reliable. Restricting ourselves to the most basic case, we investigate which coherence measures in the literature are reliability conducive. It turns out that, while a number of measures fail to be reliability conducive, except possibly in a trivial and uninteresting sense, Shogenji’s measure and several measures generated by Douven and Meijs’s recipe are notable exceptions to this rule. (shrink)

Background: Screen time among adults represents a continuing and growing problem in relation to health behaviors and health outcomes. However, no instrument currently exists in the literature that quantifies the use of modern screen-based devices. The primary purpose of this study was to develop and assess the reliability of a new screen time questionnaire, an instrument designed to quantify use of multiple popular screen-based devices among the US population. -/- Methods: An 18-item screen-time questionnaire was created to quantify use (...) of commonly used screen devices (e.g. television, smartphone, tablet) across different time points during the week (e.g. weekday, weeknight, weekend). Test-retest reliability was assessed through intra-class correlation coefficients (ICCs) and standard error of measurement (SEM). The questionnaire was delivered online using Qualtrics and administered through Amazon Mechanical Turk (MTurk). -/- Results: Eighty MTurk workers completed full study participation and were included in the final analyses. All items in the screen time questionnaire showed fair to excellent relative reliability (ICCs = 0.50–0.90; all < 0.000), except for the item inquiring about the use of smartphone during an average weekend day (ICC = 0.16, p = 0.069). The SEM values were large for all screen types across the different periods under study. -/- Conclusions: Results from this study suggest this self-administered questionnaire may be used to successfully classify individuals into different categories of screen time use (e.g. high vs. low); however, it is likely that objective measures are needed to increase precision of screen time assessment. (shrink)

Although research on the corporate social responsibility (CSR) dimension of corporate image has notably increased in recent years, the definition and measurement of the concept for academic purposes still concern researchers. In this article, literature regarding the measurement of CSR image from a customer viewpoint is revised and areas of improvement are identified. A multistage method is implemented to develop and to validate a reliable scale based on stakeholder theory. Results demonstrate the reliability and validity of this (...) new scale for measuring customer perceptions regarding the CSR performance of their service providers. With regard to this, CSR includes corporate responsibilities towards customers, shareholders, employees and society. The scale is consistent among diverse customer cohorts with different gender, age and level of education. Furthermore, results also confirm the applicability of this new scale to structural equation modelling. (shrink)

One of the major roadblocks in conducting Environmental Corporate Social Responsibility (ECSR) research is operationalization of the construct. Existing ECSR measurement tools either require primary data gathering or special subscriptions to proprietary databases that have limited replicability. We address this deficiency by developing a transparent ECSR measure, with an explicit coding scheme, that strictly relies on publicly available data. Our ECSR measure tests favorably for internal consistency and inter-rater reliability, as well as convergent and discriminant validity.

We evaluated the reliability, validity, and differential item functioning (DIF) of a shorter version of the Defining Issues Test-1 (DIT-1), the behavioral DIT (bDIT), measuring the development of moral reasoning. 353 college students (81 males, 271 females, 1 not reported; age M = 18.64 years, SD = 1.20 years) who were taking introductory psychology classes at a public University in a suburb area in the Southern United States participated in the present study. First, we examined the reliability of (...) the bDIT using Cronbach’s α and its concurrent validity with the original DIT-1 using disattenuated correlation. Second, we compared the test duration between the two measures. Third, we tested the DIF of each question between males and females. Findings reported that first, the bDIT showed acceptable reliability and good concurrent validity. Second, the test duration could be significantly shortened by employing the bDIT. Third, DIF results indicated that the bDIT items did not favour any gender. Practical implications of the present study based on the reported findings are discussed. (shrink)

A measure of coherence is said to be reliability conducive if and only if a higher degree of coherence (as measured) results in a higher likelihood that the witnesses are reliable. Recently, it has been proved that several coherence measures proposed in the literature are reliability conducive in a restricted scenario (Olsson and Schubert 2007, Synthese 157:297–308). My aim is to investigate which coherence measures turn out to be reliability conducive in the more general scenario where it (...) is any finite number of witnesses that give equivalent reports. It is shown that only the so-called Shogenji measure is reliability conducive in this scenario. I take that to be an argument for the Shogenji measure being a fruitful explication of coherence. (shrink)

Although attentional bias is considered a key characteristic of anxiety problems, the psychometric properties of most AB measures are either problematic or unknown. We conducted two experiment...

Striving for a probabilistic explication of coherence, scholars proposed a distinction between agreement and striking agreement. In this paper I argue that only the former should be considered a genuine concept of coherence. In a second step the relation between coherence and reliability is assessed. I show that it is possible to concur with common intuitions regarding the impact of coherence on reliability in various types of witness scenarios by means of an agreement measure of coherence. Highlighting the (...) need to separate the impact of coherence and specificity on reliability it is finally shown that a recently proposed vindication of the Shogenji measure qua measure of coherence vanishes. (shrink)

The direct use of a physical law for the purpose of measurement creates a problem of circularity: the law needs to be empirically tested in order to ensure the reliability of measurement, but the testing requires that we already know the value of the quantity to be measured. This problem is discussed through some detailed examples of energy measurements in quantum physics; three major methods are analyzed in their interrelation, with a focus on the method of “material (...) retardation.” It seems that the only reasonable solution is to establish the law needed for the measurement by relying on an alternate method of measurement. This solution is also circular, since it amounts to letting different measurement methods justify each other. However, this circularity can be fruitful for the process of concept building; meaning can be created in a web of interconnected measurement methods. (shrink)

The aim of this study was to check whether an online n-back task conducted in the uncontrolled environment of the Internet can yield valid and reliable data. For this purpose, 169 participants completed an online n-back task with n1, n2 and n3 blocks on their home computers. The results have shown acceptable reliability for overall accuracy and reaction time indices across n1, n2, n3 blocks, as well as for reaction time indices for each n block. Unacceptable reliability has (...) been found for separate n levels accuracy indices and for response bias indices. Confirmatory factor analysis has revealed that, among 8 proposed measurement models, the best fit for the data collected is a model with two uncorrelated factors: accuracy consisting of n1, n2, n3 indices and reaction time consisting of n2, n3 indices. The results of this study have demonstrated for the first time that a reliable administration of online n-back task is possible and may therefore give rise to new opportunities for working memory research. (shrink)

Introduction: Although deep brain stimulation often improves levodopa-responsive gait symptoms, robust therapies for gait dysfunction from Parkinson's disease remain a major unmet need. Walking speed could represent a simple, integrated tool to assess DBS efficacy but is often not examined systematically or quantitatively during DBS programming. Here we investigate the reliability and functional significance of changes in gait by directional DBS in the subthalamic nucleus.Methods: Nineteen patients underwent unilateral subthalamic nucleus DBS surgery with an eight-contact directional lead in the (...) most severely affected hemisphere. They arrived off dopaminergic medications >12 h preoperatively and for device activation 1 month after surgery. We measured a comfortable walking speed using an instrumented walkway with DBS off and at each of 10 stimulation configurations at the midpoint of the therapeutic window. Repeated measures of ANOVA contrasted preoperative vs. maximum and minimum walking speeds across DBS configurations during device activation. Intraclass correlation coefficients examined walking speed reliability across the four trials within each DBS configuration. We also investigated whether changes in walking speed related to modification of step length vs. cadence with a one-sample t-test.Results: Mean comfortable walking speed improved significantly with DBS on vs. both DBS off and minimum speeds with DBS on. Pairwise comparisons showed no significant difference between DBS off and minimum comfortable walking speed with DBS on. Intraclass correlations were ≥0.949 within each condition. Changes in comfortable walk speed were conferred primarily by changes in step length.Conclusion: Acute assessment of walking speed is a reliable, clinically meaningful measure of gait function during DBS activation. Directional and circular unilateral subthalamic DBS in appropriate configurations elicit acute and clinically significant improvements in gait dysfunction related to PD. Next-generation directional DBS technologies have significant potential to enhance gait by individually tailoring stimulation parameters to optimize efficacy. (shrink)

Standard Assessment Tasks are used in the assessment of pupil performance relative to the National Curriculum in England and Wales, assigning pupils to one of 10 levels on various Attainment Targets . This is a criterion‐referenced system, and the conventional reliability measures used in norm‐referenced systems do not apply. This paper discusses the problems involved in aggregating SA T items to give levels, and in measuring the reliability of the outcomes produced. Fitting a simple model to SAT data (...) has allowed reliability measures to be produced which are more relevant to this situation than the single numerical values applied to norm‐referenced tests. (shrink)

While we cannot directly measure the psychological preferences of individuals, and the moral, emotional, and cognitive tendencies of people from the past, we can use cultural artifacts as a window to the zeitgeist of societies in particular historical periods. At present, an increasing number of digitized texts spanning several centuries is available for a computerized analysis. In addition, developments form historical economics have enabled increasingly precise estimations of sociodemographic realities from the past. Crossing these datasets offer a powerful tool to (...) test how the environment changes psychology and vice versa. However, designing the appropriate proxies of relevant psychological constructs is not trivial. The gold standard to measure psychological constructs in modern texts – Linguistic Inquiry and Word Count – has been validated by psychometric experimentation with modern participants. However, as a tool to investigate the psychology of the past, the LIWC is limited in two main aspects: it does not cover the entire range of relevant psychological dimensions and the meaning, spelling, and pragmatic use of certain words depend on the historical period from which the fiction work is sampled. These LIWC limitations make the design of custom tools inevitable. However, without psychometric validation, there is uncertainty regarding what exactly is being measured. To overcome these pitfalls, we suggest several internal and external validation procedures, to be conducted prior to diachronic analyses. First, the semantic adequacy of search terms in bags-of-words approaches should be verified by training semantic vector spaces with the historical text corpus using tools like word2vec. Second, we propose factor analyses to evaluate the internal consistency between distinct bag-of-words proxying the same underlying psychological construct. Third, these proxies can be externally validated using prior knowledge on the differences between genres or other literary dimensions. Finally, while LIWC is limited in the analysis of historical documents, it can be used as a sanity check for external validation of custom measures. This procedure allows a robust estimation of psychological constructs and how they change throughout history. Together with historical economics, it also increases our power in testing the relationship between environmental change and the expression of psychological traits from the past. (shrink)

The concept 'Experiencing Level' points to the manner in which what a person says relates to felt experience. The manner is a first person process which is quantitatively measurable. Examples of low, middle and high Experiencing are given. In a high experiencing manner a person attends directly to a bodily sense of what is implicit and allows words to emerge from that sense. The Experiencing Scale which measures the manner of process is a third person rating of a first person (...) process, according to precise linguistic and somatic characteristics. A new rating method gives high reliability. I will briefly summarize several of the more than one hundred research studies which have used the EXP Scale or other measures of high EXP process. The high end of the EXP Scale describes what came to be called 'Focusing'. Because EXP level is a variable of the manner of process, it can be applied to almost any content area. Examples from tape-recorded psychotherapy sessions, creative writing, and theory-building will be analyzed in terms of experiencing level. Having defined the variable in observable terms makes it possible to formulate exact steps for teaching high Experiencing. Two practices -- Focusing and Thinking at the Edge -- have been developed, which can be taught in precise steps. This kind of third person variable can be found only from first person process. Its value for studying living will be shown. (shrink)

The Depression Anxiety Stress Scale-21 is an instrument in the assessment of mental health status. The current study recruited 1,532 Chinese hospital workers [74.4% female; mean age = 31.97 years] to examine the reliability, latent structure, and measurement invariance of the DASS-21 between genders. The Cronbach’s α values were greater than 0.90 for total score. This study examined four possible models of the DASS-21 using the confirmatory factor analysis in Chinese hospital workers. The results from CFA revealed that (...) the latent structure of the DASS-21 in medical staffs is best represented by a one-factor model. Then we used the one-factor model to examine measurement invariance across genders by using a multiple-group categorical CFA. All values of root mean square error approximation were less than 0.08, all Comparative Fix Index and Tucker–Lewis Index values were greater than 0.90, all ΔCFI values were less than 0.010, and ΔRMSEA were less than 0.015. These findings supported the gender invariance of the DASS-21 among Chinese hospital workers. (shrink)

This brief report assessed the psychometric validity and gender invariance of the School Burnout Inventory –a measure of students’ exhaustion, cynicism, and inadequacy– in a convenience sample of 972 high school Chilean students ranging between 12 and 18 years old. The results showed that: the SBI produces adequate scores in terms of reliability; two models fitted adequately fit to our sample and was invariant across gender; and the SBI scores were significantly related to other related constructs. Overall, the SBI (...) was found to be a reliable and valid inventory to assess school burnout in Chilean high school students. (shrink)

In _Reliable Reasoning_, Gilbert Harman and Sanjeev Kulkarni -- a philosopher and an engineer -- argue that philosophy and cognitive science can benefit from statistical learning theory, the theory that lies behind recent advances in machine learning. The philosophical problem of induction, for example, is in part about the reliability of inductive reasoning, where the reliability of a method is measured by its statistically expected percentage of errors -- a central topic in SLT. After discussing philosophical attempts to (...) evade the problem of induction, Harman and Kulkarni provide an admirably clear account of the basic framework of SLT and its implications for inductive reasoning. They explain the Vapnik-Chervonenkis dimension of a set of hypotheses and distinguish two kinds of inductive reasoning. The authors discuss various topics in machine learning, including nearest-neighbor methods, neural networks, and support vector machines. Finally, they describe transductive reasoning and suggest possible new models of human reasoning suggested by developments in SLT. (shrink)

Filling out long questionnaires can be frustrating, unpleasant, and discouraging for respondents to continue. This is why shorter forms of long instruments are preferred, especially when they have comparable reliability and validity. In present study, two short forms of the Cross-cultural Personality Assessment Inventory were developed and validated. The items of the short forms were all selected from the 28 personality scales of the CPAI-2 based on the norm sample. Based on some priori criteria, we obtained the appropriate items (...) and constructed the 56-item Chinese Personality Assessment Inventory and the 28-item CPAI. Then, we examined the factor structure of both short forms with Exploratory SEM and replicated the four-factor structure of the original CPAI-2, reflecting the four personality domains of Chinese people, namely, Social Potency, Dependability, Accommodation, and Interpersonal Relatedness. Further analyses with ESEM models demonstrate full measurement invariance across gender for both short forms. The results show that females score lower than males on Social Potency. In addition, these four factors of both short forms have adequate internal consistency, and the correlation patterns of the four factors, the big five personality traits, and several health-related variables are extremely similar across the two short forms, reflecting adequate and comparable criterion validity, convergent validity, and discriminant validity. Overall, the short versions of CPAI-2 are psychometrically acceptable and have practically implications for measuring Chinese personality and cross-cultural research. (shrink)

Measuring the effectiveness of medical interventions faces three epistemological challenges: the choice of good measuring instruments, the use of appropriate analytic measures, and the use of a reliable method of extrapolating measures from an experimental context to a more general context. In practice each of these challenges contributes to overestimating the effectiveness of medical interventions. These challenges suggest the need for corrective normative principles. The instruments employed in clinical research should measure patient-relevant and disease-specific parameters, and should not be sensitive (...) to parameters that are only indirectly relevant. Effectiveness always should be measured and reported in absolute terms (using measures such as 'absolute risk reduction'), and only sometimes should effectiveness also be measured and reported in relative terms (using measures such as 'relative risk reduction')-employment of relative measures promotes an informal fallacy akin to the base-rate fallacy, which can be exploited to exaggerate claims of effectiveness. Finally, extrapolating from research settings to clinical settings should more rigorously take into account possible ways in which the intervention in question can fail to be effective in a target population. (shrink)

PurposeThis study aims to develop and validate a stigma scale for Chinese patients with breast cancer.MethodsPatients admitted to the Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University, for breast cancer treatment participated in this study. Development of the Breast Cancer Stigma Scale involved the following procedures: literature review, interview, and applying a theoretical model to generate items; the Breast Cancer Stigma Scale’s content validity was assessed by a Delphi study and feedback from patients with breast cancer ; (...) exploratory factor analysis was used to assess the construct validity; convergent validity was assessed with the Social Impact Scale ; internal consistency Cronbach’s α, split-half reliability, and test–retest reliability were used to identify the reliability of the scale.ResultsThe final version of the Breast Cancer Stigma Scale consisted of 15 items and showed positive correlations with the Social Impact Scale. Exploratory factor analysis revealed four components of the Breast Cancer Stigma Scale: self-image impairment, social isolation, discrimination, and internalized stigma, which were strongly related to our perceived breast cancer stigma model and accounted for 69.443% of the total variance. Cronbach’s α for the total scale was 0.86, and each subscale was 0.75–0.882. The test–retest reliability with intra-class correlation coefficients of the total scale was 0.947, and split-half reliability with intra-class correlation coefficients of the total scale was 0.911. The content validity index was 0.73–1.0.ConclusionThe newly developed Breast Cancer Stigma Scale offers a valid and reliable instrument for assessing the perceived stigma of patients with breast cancer in clinical and research settings. It may be helpful for stigma prevention in China. (shrink)

The neo-Kohlbergian approach to moral reasoning development maintains that intermediate concepts lie between bedrock moral schemas and professional codes of ethics and deal with issues of confidentiality, competence, informed consent, allocation of resources and professional autonomy (Rest et. al, 1999b Rest, J., Narvaez, D., Bebeau, M., & Thoma, S. J. (1999b). Postconventional moral thinking: A neo-Kohlbergian approach. Lawrence Erlbaum Associates.[Crossref], [Google Scholar]). Intermediate concepts provide concrete guides for behavior, they contrast general moral schemas that are concerned with issues of fairness, (...) justice and equality. This paper outlines the development and testing of a prototype Intermediate Concept Measure for teachers (TICM). Results indicate that the TICM has respectable psychometric properties and assesses a construct in the moral domain. (shrink)

Science is remarkably reliable. It puts people on the moon, performs laser eye surgery, tells us about ancient civilisations and species, and predicts the future of our climate. What underwrites this reliability? This book argues that the standard answers—the scientific method, rigour, and objectivity—are insufficient for the job. Here we propose a new model of science that places its products front and centre. This is the ‘Tangle of Science’. In this book we show how any reliable piece of science (...) is underpinned by a vast, diverse, and thick network of other scientific products. In doing so we bring back into focus areas of science that have been long neglected, emphasising how every product, from the screws that hold the space shuttle together to ways of measuring the consumer price index to Einstein’s theory of general relativity, work together to support results we can trust. (shrink)