Elman (2009) proposed that the traditional role of the mental lexicon in language processing can largely be replaced by a theoretical model of schematic event knowledge founded on dynamic context-dependent variables. We evaluate Elman’s approach and propose an alternative view, based on dual coding theory and evidence that modality-specific cognitive representations contribute strongly to word meaning and language performance across diverse contexts which also have effects predictable from dual coding theory.

The principal temptation toward substance dualisms, or otherwise incorporating a question begging homunculus into our psychologies, arises not from the problem of consciousness in general, nor from the problem of intentionality, but from the question of our awareness and understanding of our own mental contents, and the control of the deliberate, conscious thinking in which we employ them. Dennett has called this "Hume's problem". Cognitivist philosophers have generally either denied the experiential reality of thought, as did the Behaviorists, or have (...) taken an implicitly epiphenomenalist stance, a form of dualism. Some sort of mental duality may indeed be required to meet this problem, but not one that is metaphysical or question begging. I argue that it can be solved in the light of Paivio's "Dual Coding" theory of mental representation. This theory, which is strikingly simple and intuitive (perhaps too much so to have caught the imagination of philosophers) has demonstrated impressive empirical power and scope. It posits two distinct systems of potentially conscious representations in the human mind: mental imagery and verbal representation (which is not to be confused with 'propositional' or "mentalese" representation). I defend, on conceptual grounds, Paivio's assertion of precisely two codes against interpretations which would either multiply image codes to match sense modes, or collapse the two, admittedly interacting, systems into one. On this basis I argue that the inference that a conscious agent would be needed to read such mental representations and to manipulate them in the light of their contents can be pre-empted by an account of how the two systems interact, each registering, affecting and being affected by developing associative processes within the other. (shrink)

In light of the dramatic growth of Chinese learners worldwide and a need for cross-linguistic research on Chinese literacy development, this study drew upon theories of visual complexity effect (Su & Samuels, 2010) and dual-coding processing (Sadoski & Paivio, 2013) and investigated a) the effects of character properties (i.e., visual complexity and radical presence) on character acquisition and b) the relationship between individual learner differences in radical awareness and character acquisition. Participants included adolescent English-speaking beginning learners of Chinese in the (...) U.S. Following Kuo et al. (2014), a novel character acquisition task was used to investigate the process of acquiring the meaning of new characters. Results showed that a) characters with radicals and with less visual complexity were easier to acquire than characters without radicals and with greater visual complexity; and b) individual differences in radical awareness were associated with the acquisition of all types of characters, but the association was more pronounced with the acquisition of characters with radicals. Theoretical and practical implications of the findings were discussed. (shrink)

Rapid development of e-learning courses for ethics-and-compliance programs led to substantial success in producing engaging multimedia training toolkits aimed at breaking through barriers of indifference and distrust by combining learning with fun. However, a pleasant training experience is no guarantee of its ultimate success in improving organizational ethics. Drawing on Paivio’s Dual Coding Theory, this paper presents a model for evaluating multimedia learning from a moral viewpoint. The main argument advanced in the paper is that entertaining multimedia training modules, as (...) commonly offered in the business world, may deliver an unintended message which runs counter to the very goal of ethics training. The Rocked or Shocked? videogame—a multimedia-based ethics training module—will serve as a case study for detecting the complicated role of entertainment in ethics training and examining the ethical meanings conveyed through the pictorial/event channel. The paper concludes with recommendations for the design of multimedia ethics training. (shrink)

Rapid development of e-learning courses for ethics-and-compliance programs led to substantial success in producing engaging multimedia training toolkits aimed at breaking through barriers of indifference and distrust by combining learning with fun. However, a pleasant training experience is no guarantee of its ultimate success in improving organizational ethics. Drawing on Paivio’s Dual Coding Theory, this paper presents a model for evaluating multimedia learning from a moral viewpoint. The main argument advanced in the paper is that entertaining multimedia training modules, as (...) commonly offered in the business world, may deliver an unintended message which runs counter to the very goal of ethics training. The Rocked or Shocked? videogame—a multimedia-based ethics training module—will serve as a case study for detecting the complicated role of entertainment in ethics training and examining the ethical meanings conveyed through the pictorial/event channel. The paper concludes with recommendations for the design of multimedia ethics training. (shrink)

Chapter 1 First Person Access to Mental States. Mind Science and Subjective Qualities -/- Abstract. The philosophy of mind as we know it today starts with Ryle. What defines and at the same time differentiates it from the previous tradition of study on mind is the persuasion that any rigorous approach to mental phenomena must conform to the criteria of scientificity applied by the natural sciences, i.e. its investigations and results must be intersubjectively and publicly controllable. In Ryle’s view, philosophy (...) of mind needs to adopt an antimentalist stance to achieve this aim. Antimentalism not only definitively rejects the idea that mind is a substance separated from the body, it also denies that mental phenomena radically differ from physical phenomena by virtue of several unique features. Most problematically, mental phenomena have a conscious character (mental states are related to specific qualitative feelings) and are accessible only to the first-person (only the subject knows directly what s/he is experiencing inside his/her mind). Ryle takes a strong stance on antimentalism going so far as to maintain that an approach to mind which aims to meet the criteria of scientificity set by the natural sciences must avoid any reference to internal, unobservable mental states. In his view (which is considered a specifically philosophical version of psychological behaviorism and also addresses questions put forward in psychological research), mental states can be redescribed in terms of behavioral dispositions. In this chapter, we address the historical roots of the antimentalist view and analyze its relation to the later tradition of research on mind. We show that, compared to the antimentalist stance, functionalism and cognitivism take a step back when they maintain that direct reference to mental states is necessary since mental states cause and therefore explain human behavior. This step backwards is often interpreted as a return to mentalism. However, this is only partially true. Indeed, we suggest that these later traditions retain one important element of Ryle’s antimentalism, i.e. the idea that mental states must be uniquely identified using external and publicly observable criteria, while excluding any reference to introspection and those qualitative dimensions of a mental state, which are accessible only to the first-person. According to the perspective we put forward, this epistemological stance has continued to influence contemporary research on mind and current philosophical and psychological theories which both tend to exclude the subjective qualities of human experience from their accounts of how the mind works. The issue we raise here is whether this is legitimate or whether subjective qualities do play a role with respect to the way our mind works. The conclusion of this chapter anticipates the argument the book makes in in favor of this latter position, starting from a particular angle, i.e. the problem of how we categorize concepts related to our internal states. -/- Chapter 2 The Misleading Aspects of the Mind/Computer Analogy. The Grounding Problem and the Thorny Issue of Propriosensitive Information -/- Abstract. After the crisis of behaviorism, cognitivism and functionalism became the predominant models in the field of psychology and of philosophy, respectively. Their success is mainly due to the new key they use for interpreting mental processes: the mind/computer analogy. On the basis of this analogy, mental operations are seen as cognitive processes based on computations, i.e. on manipulations of abstract symbols which are in turn understood as informational unities (representations). This chapter identifies two main problems with this model. The first is how these symbols can relate to and communicate with perception and thus allow us to identify and classify what we perceive through the senses. Here we limit ourselves to presenting this issue in relation to the classical symbol grounding problem originally put forward by Harnad on the basis of Searle’s Chinese room argument. An attempt to address the problem raised here will be made in chap. 3. The second point we discuss in relation to the mind/computer analogy concerns the idea of information it fosters. Indeed, following this analogy, information is something available in the external world which can be captured by the senses and transmitted to the central system without being influenced or modified by the procedures of transmission. This perspective does not take into account that – unlike computers – in living beings information is acquired by means of the body. As Ulric Neisser has already pointed out, the body is itself an informational source that provides us with additional sensory experience that influences (modifies or complements) the information extracted from the external world by the senses. To develop this line of analysis and to determine exactly what information is provided by the body and how this might influence cognition, we examine Sherrington’s and Gibson’s positions. Moving on from their views, we qualify bodily information in terms of ‘proprioception’. We use ‘proprioception’ in a broad sense to describe any kind of experience we have of our internal states (including postural information as well as sensations related to the general state of the body and its parts). Following Damasio’s and Craig’s studies, we further elaborate this position, arguing that living beings are equipped with an internal propriosensitive monitoring system which maps all the changes that constantly occur in our body and that give us perceptual (‘proprioceptive’ or propriosensitive) information about what happens inside us. Moreover, relying on Goldie’s and Ratcliffe’s view, we show that emotional information can also be considered as a form of ‘proprioception’ which contributes to determining everything we perceive. This analysis leads us to the second main thesis of this book: ‘proprioception’ is a form of internal perception and it is an essential component of the sensory information we can access and use for all cognitive purposes. -/- Chapter 3 Semantic Competence from the Inside: Conceptual Architecture and Composition -/- Abstract. Concepts are essential constituents of thought: they are the instruments we use to categorize our experience, i.e. to classify things and group them together in homogeneous sets. Here we define concepts as the internal mental information (representations) that allows us, among other things, to master words in natural language. By analyzing the way in which individuals master word meanings we explore a number of hypotheses regarding the nature of concepts. Following Diego Marconi’s research, we differentiate between two kind of abilities that underpin lexical competence – so-called ‘referential’ and ‘inferential competence’ – and we suggest that, in order to support these abilities, concepts must also include two corresponding kinds of information, i.e. inferential and referential information. We point out that the most widely used and acknowledged theories of concepts do not make this distinction, instead broadly characterizing the information used for categorization in terms of propositionally described feature lists. However, we show that while feature lists can explain inferential competence, they do not account for referential competence. To address the issue of referential competence we examine Ray Jackendoff’s hypothesis that to account for the possibility of linking perceptual and conceptual information we need to assume the existence of a (visual) representation that encodes the geometric and topological properties of objects and bridges the gap between the percept and the concept. Furthermore, we analyze the extension of this work by Jesse Prinz who introduced the notion of a proxytype, a perceptual representation of a class of objects that incorporates structural and parametric information related to their appearance. However, as we point out, proxytypes can only explain the relationship between perception and concepts with respect to instances that can be perceived through the senses and that belong to the same class by virtue of their physical similarity. We suggest that this notion be extended to include larger conceptual classes. To accomplish this, we further develop Mark Johnson, George Lakoff and Jean Mandler’s idea of a schematic image and argue that conceptual representations include a perceptual schema. Perceptual schemata are non-linguistic, structured experiential gestalts (patterns or maps) that make use of information taken from all sensory modalities, including body perception. They accomplish a quasi-conceptual function: they allow us to recognize and to classify different instances. In this work, we hypothesize that perceptual schemata are an essential component of concepts, but not identical to them. Instead, we suggest that concepts include both perceptual and propositional information with perceptual schemata providing the ‘perceptual core concept’ that grounds related propositional information. -/- Chapter 4 In the Beginning There Were Categories. The Bodily Origin of Prelinguistic Categorical Organization: The Example of Folkbiological Taxonomies -/- Abstract. Studies of categorization in psychology and the cognitive sciences have made use of the notions of ‘category’ and ‘concept’ without precisely defining what is meant by either; in fact, often these terms have been used as synonyms, making it difficult to address specific issues related to conceptual development. This chapter begins by discussing the definitions of, as well as the distinctions between, ‘categories’ and ‘concepts’ in the classical philosophical tradition (Aristotle, Kant and Husserl). We introduce our view of categories with reference to Husserl. Categorization is defined as the way in which our experience is originally (pre-linguistically) organized in a passive and fully unconscious manner on the basis of universal structuring principles. Conceptualization is explained as a later process in which the earlier categorical macro-classes are further subdivided into more specific and detailed sets; this later process also relies on linguistic learning and exposure to culture. On the basis of Ray Jackendoff, Jean Mandler, George Lakoff and Mark Johnson’s work, we hypothesize that categorization is a two-step process that begins with the formation of homogeneous sets of entities partitioned into regions that describe the ontological boundaries of the objects that humans perceive (categories) and continues at a later stage with the development of more specific classifications (concepts). In this chapter, we mainly address three related issues: Why should we assume that there is categorical organization which precedes the development of a conceptual system? How do categories and concepts relate to each other? Shall we hypothesize that categories are innate or that they are formed before concepts on the basis of information and organizational structure available at a very early developmental stage? We show that categorical partitions are necessary for categorization and, following Mandler, that the general categories we form at an early age do not match our adult superordinate concepts. As for the third issue, we argue that there might be no need to assume that categories are innately present in the human mind, since their formation can be explained – at least in certain cases – by basic mechanisms that work on body (propriosensitive) information. This hypothesis will not be discussed in general, but in relation to a particularly relevant example of categorical partition, i.e. the folk-biological dichotomy between ANIMATE/INANIMATE. This is compared with other dichotomies that derive from it, but are not directly categorical such as LIVING/NON-LIVING and BIOLOGICAL/NON-BIOLOGICAL. -/- Chapter 5 Internal States: From Headache to Anger. Conceptualization and Semantic Mastery -/- Abstract. Here we ask if we can also apply the distinction between referential and inferential competence we introduced in Chapter 3 to words that do not refer to things that are perceived using the external senses, especially to words/concepts that denote bodily experiences (such as pain, thirst, hunger, etc.) or emotions. We introduce and discuss the hypothesis that – even though such words/concepts do not refer to intersubjectively identifiable entities in the external world – they do have a kind of referent that can be accessed via direct perception, more specifically ‘proprioception’, as we have defined it in terms of all propriosensitive information we can consciously access. In the first part of the chapter, we specifically consider terms denoting bodily experiences such as ‘pain’ or ‘hunger’ and argue that their referents are identified and classified from a first-personal point of view on the basis of four main characteristics: their specific intensity, their localization in the body, their co-occurrence with other signals and above all their specific qualitative sensations. Emotions are addressed in the second part of the chapter. We suggest that there is a continuity between bodily experiences and emotions. In particular, we argue for a perceptual theory of emotions in line with that proposed by James and Lange at the end of the 19th century and developed more recently by authors such as Damasio (see also chap. 2§5, §6). The hypothesis we put forward is that the referential information that supports the categorization of emotions and therefore also our mastery of terms referring to emotions consists in the perception (i.e. the ‘proprioception’) of those bodily states and changes in bodily states which constitute our emotional experience. In the context of this discussion we examine some objections to this line of reasoning that arise from a cognitivist perspective and following authors such as Oatley, Johnson-Laird and Frijda, we distinguish between basic emotions that can be identified and classified solely on the basis on how they feel and complex emotions whose identification and classification additionally depends on cognitive factors. To describe how emotions are identified and classified on the basis of how they feel, we rely on Marcel and Lambie’s distinction between an ‘emotion state’ and an ‘emotion experience’. Both notions indicate kinds of feelings that we consciously experience. However, they describe first-order and second order emotion awareness respectively. The emotion state is the feeling we have of the bodily states and changes that occur when we are experiencing an emotion, while the emotion experience is the fully developed and integrated emotion we both experience and are, with reflection, aware of experiencing. On the basis of this differentiation, we also show that the same characteristics that aid in the identification and classification of bodily experiences (specific qualitative sensations; somatic localization; specific intensity; presence/absence of specific concomitant sensations) can also be used for the identification and classification of emotions – at least basic emotions. In the last two sections of the chapter we present some clinical evidence on the semantic competence of people who suffer from Alexithymia and Autism Spectrum Disorder which supports the conclusions of our preceding analyses. -/- Chapter 6 The ‘Proprioceptive’ Component of Abstract Concepts -/- Abstract. In this chapter, we address the issue of whether the mastery of abstract words requires only inferential knowledge and thus, if the concepts that support the mastery of abstract words include only linguistic information. We start by differentiating the notions of ‘abstract’ and ‘general’ which are often erroneously confused. We then identify a strict definition of abstract, as contrasted with ‘concrete’, that applies to words or concepts whose referent cannot be experienced by the senses. We argue that abstract words/concepts would be better described as theoretical, because they are usually conceived as structured sets of inferential knowledge expressed linguistically; that is, as small theories. Pointing out parallels with Carnap’s analysis of this issue in philosophy of science, we hypothesize that words/concepts denoting non-observable entities are not all ‘equally theoretical’, because their link to sensory experience can be stronger or weaker. We revive the distinction, inspired by Quine, between theoretical and intertheoretical concepts/words. This distinction relies on the fact that the former – unlike the latter – retains a strong, although indirect, connection with perception. In Quine’s discussion, perception is understood uniquely in terms of observability, i.e. of external sensory experience. Here we argue, however, that bodily, ‘proprioceptive’ (i.e. propriosensitive) experience can also serve to referentially ground theoretical (i.e. abstract) concepts/words. We frame this issue using the example of the theoretical concept ‘freedom’ and Lakoff’s hypothesis that this concept is developed on the basis of bodily information. We contrast this with the example of ‘democracy’ which more closely resembles an intertheoretical concept/word. Furthermore, we show that one of the classical views put forward in psycholinguistic research to explain how abstract concepts are mentally represented – i.e. Paivio’s Dual Coding Theory – points in the same direction as our analysis. The same is true of Barsalou’s work suggesting that we use internal information to understand at least some abstract words. To sustain this position, we put forward two lines of evidence: the first comes from psycholinguistic studies while the second examines deficits of semantic competence exhibited by people with Autism Spectrum Disorder. On the basis of our analysis, we put forward a classification that distinguishes between different kinds of concreteness and different degrees of abstraction: concepts/words referring to body experiences and basic emotions are described as analogous to concrete concepts/words because they are grounded in perceptual (i.e. propriosensitive) experience, while abstract concepts/words are considered more or less abstract depending on whether they are intratheoretical (and rely entirely on inferential information) or theoretical (and are partially grounded in perceptual – or more often in propriosensitive perceptual – information). In the last section of the chapter we consider two scales that have been used in psycholinguistic research to measure the degree of concreteness vs. abstractness of words and we show that – used conjointly – they can provide a measure of the internal vs. external grounding of specific words. (shrink)