We argue that formal linguistic theory, properly extended, can provide a unifying framework for diverse phenomena beyond traditional linguistic objects. We display applications to pictorial meanings, visual narratives, music, dance, animal communication, and, more abstractly, to logical and non-logical concepts in the ‘language of thought’ and reasoning. In many of these cases, a careful analysis reveals that classic linguistic notions are pervasive across these domains, such as for instance the constituency (or grouping) core principle of syntax, the (...) use of logical variables (for object tracking), or the variety of inference types investigated in semantics/pragmatics. The aim of this overview is to show how the application of formal linguistic concepts and methodology to non-linguistic objects yields non-trivial insights, thus opening the possibility of a general, precise theory of signs. (An appendix, found in the online supplements to this article, surveys applications of Super Linguistics to animal communication.). (shrink)

We argue that formal linguistic theory, properly extended, can provide a unifying framework for diverse phenomena beyond traditional linguistic objects. We display applications to pictorial meanings, visual narratives, music, dance, animal communication, and, more abstractly, to logical and non-logical concepts in the ‘language of thought’ and reasoning. In many of these cases, a careful analysis reveals that classic linguistic notions are pervasive across these domains, such as for instance the constituency (or grouping) core principle of syntax, the (...) use of logical variables (for object tracking), or the variety of inference types investigated in semantics/pragmatics. The aim of this overview is to show how the application of formal linguistic concepts and methodology to non-linguistic objects yields non-trivial insights, thus opening the possibility of a general, precise theory of signs. (An appendix, found in the online supplements to this article, surveys applications of Super Linguistics to animal communication.). (shrink)

As part of a recent attempt to extend the methods of formal semantics beyond language, it has been claimed that music has an abstract truth-conditional semantics, albeit one that has more in common with iconic semantics than with standard compositional semantics. After summarizing this approach and addressing a common objection, we argue that music semantics should be enriched in three directions by incorporating insights of other areas of Super Semantics. First, it has (...) been claimed by Abusch 2013 that visual narratives make use of discourse referents akin to those we find in language. We argue that a similar conclusion extends to music, and we highlight it by investigating ways in which orchestration and dance may make cross-referential dependencies more explicit. Second, we show that by bringing music semantics closer to the semantics of visual narratives, we can give an account of the semantics of mixed visual and musical sequences. Third, it has been claimed that co-speech gestures trigger characteristic conditionalized presuppositions, called ‘cosuppositions’, and that their semantic status derives from their parasitic character relative to words. We argue that the same conclusion extends to some instances of film and cartoon music: it may trigger cosuppositions that can be revealed by embedding film excerpts or gifs in sentences so as to test presupposition projection. We further argue that under special discourse conditions, pro-speech gestures and pro-speech music alike can trigger cosuppositions as well. These results suggest that new insights can be gained not just by extending the methods of semantics to new objects, but also by drawing new connections among them. (shrink)

Recent advances in the Super Linguistics of pictures have laid the Super Semantic foundation for modelling the phenomena of narrative sequencing and co-reference in pictorial and mixed linguistic-pictorial discourses. We take up the question of how one arrives at the pragmatic interpretations of such discourses. In particular, we offer an analysis of: (i) the discourse composition problem: how to represent the joint meaning of a multi-picture discourse, (ii) observed differences in narrative sequencing in prima facie equivalent (...) linguistic vs pictorial discourses, and (iii) the phenomenon of co-referencing across pictures. We extend Segmented Discourse Representation Theory to spell out a formal Super Pragmatics that applies to linguistic, pictorial and mixed discourses, while respecting the particular ‘genius’ of either medium and computing their distinctive pragmatic interpretations. (shrink)

A language is a system of signs used for communication, and linguists are tasked with, among other things, uncovering the syntax and semantics of such systems. In this paper I explore to what extent pictures fit this characterization of a language and hence would fall within the domain of linguistics. I conclude that at the very least there are well-defined systems of depiction for which we can give a precise semantics, in a familiar possible worlds framework, although (...) class='Hi'>pictorial propositions are not derived via the linguistically familiar lexicon and recursive composition rules, but via geometric projection. I then show how sequences of pictures, like sequences of utterances, can be used to form coherent discourses. I explore in some detail extensions of the formal discourse semantics of Segmented Discourse Representation Theory to deal with comics, i.e. storytelling through picture sequences. I focus specifically on the representation of events in comics, and the integration of both symbolic and iconic elements. (shrink)

Pictorial free perception reports are sequences in comics or film of one unit that depicts an agent who is looking, and a following unit that depicts what they see. This paper proposes an analysis in possible worlds semantics and event semantics of such sequences. Free perception sequences are implicitly anaphoric, since the interpretation of the second unit refers to the agent depicted in the first. They are argued to be possibly non-extensional, because they can depict hallucination or (...) mis-perception. The semantics proposed here employs an account of anaphora using discourse referents, a formalized possible worlds semantics for pictorial narratives, and, to model the epistemic consequences of perceptual events, the event alternative construction from dynamic epistemic logic. In intensional examples, the second unit depicting what is seen is analyzed as embedded. It is argued that a semantics for embedding where the attitudinal state of the depicted agent is required to entail the semantic content of the picture attributes too much information to the agent. This is addressed with a model of normal looking, and a semantics for the embedding construction that uses existential quantification over alternatives, rather than universal quantification. (shrink)

Semantics traditionally focuses on linguistic meaning. In recent years, the Super Linguistics movement has tried to broaden the scope of inquiry in various directions, including an extension of semantics to talk about the meaning of pictures. There are close similarities between the interpretation of language and of pictures. Most fundamentally, pictures, like utterances, can be either true or false of a given state of affairs, and hence both express propositions (Zimmermann, 2016; Greenberg, 2013; Abusch, 2015). Moreover, sequences (...) of pictures, like sequences of utterances, can be used to tell stories. Wordless picture books, comics, and film are cases in point. In this paper I pick up the project of providing a dynamic semantic account of pictorial story-telling, started by Abusch (2012) and continued by Abusch & Rooth (2017); Maier & Bimpikou (2019); Fernando (2020). More specifically, I propose here a semantics of speech and thought bubbles by adding event reference to PicDRT. To get there I first review the projection-based semantics for pictures (section 1), noting the fundamental distinction between symbolic and iconic meaning that makes speech bubbles especially interesting (section 2). I then review the dynamic PicDRT framework for pictorial narratives (section 3), add events (section 4), and propose an account of speech bubbles as quotational event modification (section 5). I end with a brief look at other conventional, symbolic enrichments in comics (section 6). (shrink)

The iconic revolution changing the routine of everyday communication is gradually leading to the creation of a linguistic structure that combines visual and verbal tools in both formal and semantic aspects. Computer and mobile applications today enable high-tech imaging that ensures the spread of iconic communication in mundane interactions and the possibility of a creative combination of verbal and iconic codes for language users who navigate in a world of images in an increasingly confident manner. The iconic revolution (...) that accompanies this expansion of new communication technologies thus leads to serious changes in language use, thereby enhancing the transmission of verbal and iconic language to become a key element of mundane communication. This article argues that this turn in communication technology guided the attention of Kristóf Nyíri, an eminent figure in contemporary Hungarian philosophy, to the problem of the relationship between icon, language, and tradition. The aim of the present essay is twofold: to shed light on the relevance of Nyíri’s analysis and thoughts on the relationship between tradition and language in the history of communication, and to identify, using Nyíri’s model of pictorial meaning as a starting point, the relevant aspects of the conception of technological determinism in answering questions from the philosophy of language and mind. (shrink)

We argue that the pictorial nature of certain constructions in signs and in gestures explains surprising properties of their syntax. In several sign languages, the standard word order (e.g. SVO) gets turned into SOV (with preverbal arguments) when the predicate is a classifier, a distinguished construction with highly iconic properties (e.g. Pavlič, 2016). In silent gestures, participants also prefer an SOV order in extensional constructions, irrespective of the word order of the language they speak (Goldin-Meadow et al., 2008). But (...) in silent gestures and in Brazilian Sign Language (Libras), intensional constructions can override these SOV preferences, yielding SVO instead (Schouwstra & de Swart, 2014; Napoli et al., 2017). This distinction was argued to be due to iconicity: arguments are expressed before the verb if they correspond to entities that are present before the action, otherwise they follow the verb. While agreeing with this intuition, we propose that the extensional/intensional distinction is neither empirically nor theoretically appropriate. In new data from American Sign Language, we replicate the distinction among extensional classifier predicates: for _x ate up the ball_, the ball is typically seen before the eating and a preposed object is preferred; but for _x spit out the ball_, the ball is typically seen after the spitting and a postposed object is preferred, although both _eat up_ and _spit out_ are used extensionally. We extend this finding to data involving pro-speech (= speech-replacing) gestures embedded in French sentences. We argue for a Visibility Generalization according to which arguments appear before the verb if their denotations are typically visible before the action, and we develop a new formal account within a pictorial semantics for visual animations (inspired by Greenberg and Abusch). It derives the observed word order preferences, it explains how the semantics of classifier predicates combines iconic and conventional properties, and it makes a more general point: sign language semantics combines logical semantics with pictorial semantics. (shrink)

Visual narratives like comics involve a range of complex cognitive operations in order to be understood. The Parallel Interfacing Narrative‐Semantics (PINS) Model integrates an emerging literature showing that comprehension of wordless image sequences balances two representational levels of semantic and narrative structure. The neurocognitive mechanisms that guide these processes are argued to overlap with other domains, such as language and music.

This book attempts to marry truth-conditional semantics with cognitive linguistics in the church of computational neuroscience. To this end, it examines the truth-conditional meanings of coordinators, quantifiers, and collective predicates as neurophysiological phenomena that are amenable to a neurocomputational analysis. Drawing inspiration from work on visual processing, and especially the simple/complex cell distinction in early vision (V1), we claim that a similar two-layer architecture is sufficient to learn the truth-conditional meanings of the logical coordinators and logical quantifiers. As (...) a prerequisite, much discussion is given over to what a neurologically plausible representation of the meanings of these items would look like. We eventually settle on a representation in terms of correlation, so that, for instance, the semantic input to the universal operators (e.g. and, all)is represented as maximally correlated, while the semantic input to the universal negative operators (e.g. nor, no)is represented as maximally anticorrelated. On the basis this representation, the hypothesis can be offered that the function of the logical operators is to extract an invariant feature from natural situations, that of degree of correlation between parts of the situation. This result sets up an elegant formal analogy to recent models of visual processing, which argue that the function of early vision is to reduce the redundancy inherent in natural images. Computational simulations are designed in which the logical operators are learned by associating their phonological form with some degree of correlation in the inputs, so that the overall function of the system is as a simple kind of pattern recognition. Several learning rules are assayed, especially those of the Hebbian sort, which are the ones with the most neurological support. Learning vector quantization (LVQ) is shown to be a perspicuous and efficient means of learning the patterns that are of interest. We draw a formal parallelism between the initial, competitive layer of LVQ and the simple cell layer in V1, and between the final, linear layer of LVQ and the complex cell layer in V1, in that the initial layers are both selective, while the final layers both generalize. It is also shown how the representations argued for can be used to draw the traditionally-recognized inferences arising from coordination and quantification, and why the inference of subalternacy breaks down for collective predicates. Finally, the analogies between early vision and the logical operators allow us to advance the claim of cognitive linguistics that language is not processed by proprietary algorithms, but rather by algorithms that are general to the entire brain. Thus in the debate between objectivist and experiential metaphysics, this book falls squarely into the camp of the latter. Yet it does so by means of a rigorous formal, mathematical, and neurological exposition – in contradiction of the experiential claim that formal analysis has no place in the understanding of cognition. To make our own counter-claim as explicit as possible, we present a sketch of the LVQ structure in terms of mereotopology, in which the initial layer of the network performs topological operations, while the final layer performs mereological operations. The book is meant to be self-contained, in the sense that it does not assume any prior knowledge of any of the many areas that are touched upon. It therefore contains mini-summaries of biological visual processing, especially the retinocortical and ventral /what?/ parvocellular pathways computational models of neural signaling, and in particular the reduction of the Hodgkin-Huxley equations to the connectionist and integrate-and-fire neurons Hebbian learning rules and the elaboration of learning vector quantization the linguistic pathway in the left hemisphere memory and the hippocampus truth-conditional vs. image-schematic semantics objectivist vs. experiential metaphysics and mereotopology. All of the simulations are implemented in MATLAB, and the code is available from the book’s website. • The discovery of several algorithmic similarities between visison and semantics. • The support of all of this by means of simulations, and the packaging of all of this in a coherent theoretical framework. (shrink)

The specific goal of the article is to investigate the principles governing the perception of rhythmic structure in dance and music—taken separately and together—on the basis of a case study. I take as a starting point Lerdahl and Jackendoff’s (A generative theory of tonal music. MIT Press, 1983) conception of musical rhythm as the interaction between grouping and meter, and I examine to what extent it can apply to dance. Then, I explore how the rhythmical structures of music and dance (...) interact in a single event. I conclude that dance and music perception largely share the same abstract system, and the differences in the properties of their structure derives from the different (visual vs. auditory) modalities in which they are perceived; their modality difference also affects the perceived structure resulting from their combination in dance-music events. The exploration is guided by a detailed examination of the opening of Stravinsky’s _Augurs of Spring_ (1913) as choreographed by Nijinsky (1913), Béjart (1970) and Bausch (1975). By comparing these minimal pairs of dance-music events, I adapt the formal methodology of linguistics to other cognitive systems. The general goal of the article is to shed further light on the organizational principles of mental representations by comparing several cognitive systems in order to distinguish between general cognitive properties and modality-specific or domain-specific properties. (shrink)

We propose an extension of Discourse Respresentation Theory (DRT) for analyzing pictorial narratives. We test drive our PicDRT framework by analyzing the way authors represent characters’ mental states and perception in comics. Our investigation goes beyond Abusch and Rooth (2017) in handling not just free perception sequences, but also a form of apparent perspective blending somewhat reminiscent of free indirect discourse.

Arbitrary symbolism is a linguistic doctrine that predicts an orthogonal relationship between word forms and their corresponding meanings. Recent corpora analyses have demonstrated violations of arbitrary symbolism with respect to concreteness, a variable characterizing the sensorimotor salience of a word. In addition to qualitative semantic differences, abstract and concrete words are also marked by distinct morphophonological structures such as length and morphological complexity. Native English speakers show sensitivity to these markers in tasks such as auditory word recognition and naming. One (...) unanswered question is whether this violation of arbitrariness reflects an idiosyncratic property of the English lexicon or whether word concreteness is a marked phenomenon across other natural languages. We isolated concrete and abstract English nouns, and translated each into Russian, Arabic, Dutch, Mandarin, Hindi, Korean, Hebrew, and American Sign Language. We conducted offline acoustic analyses of abstract and concrete word length discrepancies across languages. In a separate experiment, native English speakers with no prior knowledge of these foreign languages judged concreteness of these nouns. Each naïve participant heard pre-recorded words presented in randomized blocks of three foreign languages following a brief listening exposure to a narrative sample from each respective language. Concrete and abstract words differed by length across five of eight languages, and prediction accuracy exceeded chance for four of eight languages. These results suggest that word concreteness is a marked phenomenon across several of the world's most widely spoken languages. We interpret these findings as supportive of an adaptive cognitive heuristic that allows listeners to exploit non-arbitrary mappings of word form to word meaning. (shrink)

Situation theory is the result of an interdisciplinary effort to create a full-fledged theory of information. Created by scholars and scientists from cognitive science, computer science, AI, linguistics, logic, philosophy, and mathematics, the theory is forging a common set of tools for the analysis of phenomena from all these fields. This volume presents work that evolved out of the Second Conference on Situation Theory and its Applications. Twenty-six essays exhibit the wide range of the theory, covering such topics (...) as natural language semantics, philosophical issues about information, mathematical applications, and the visual representation of information in computer systems.Jon Barwise is a professor of philosophy, mathematics, and logic at Indiana University in Bloomington. Jean Mark Gawron is a researcher at SRI International and a consultant at Hewlett-Packard Laboratories. Gordon Plotkin is a professor of theoretical computer science at the University of Edinburgh. Syun Tutiya is in the philosophy department at Chiba University in Japan. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:Gerald Prince WORLDS WITH STYLE Whether it is taken to be a laudable characteristic of verbal artifacts (as in, "This essay is really well written"), a distinctive feature of an individual manner of speaking or writing (as in, "Jane definitely has a style of her own"), an ornamental supplement to that which is expressed (style as elocutio), or an appropriate way of using language in different contexts (there is (...) — or was — a style for tragedy, another one for comedy, and still another one for pastoral; there is a style for speaking at business meetings and other styles better suited to everyday conversations, political rallies, or Ph.D. exams), whether it is taken to be any of these four things (or many other things as well), the style of an utterance is traditionally viewed as a secondary phenomenon different from and added to some primary base. It is the mode of expressing a given matter, the way of "saying" some thing, the how of a what. The same is true when the notion style is used in connection with nonverbal artifacts — we talk about musical styles and about pictorial ones — but also in connection with any number of activities or states of affairs: walking down the street, dancing the tango, being pleasant, or playing ping-pong. Ping-pong players use rackets, a net, and a ball, in accordance with certain rules, when they engage in a casual game; but they are thought to be doing somediing more than merely playing: they are performing an activity (the what) in a particular way. They do it stylishly, idiosyncratically, with a lot of showboating, or in a manner appropriate to the informal context.1 It is somewhat surprising, therefore, that, when it comes to verbal (and nonverbal) representations, style is often thought to be irrelevant to the signified (what is represented) as opposed to the signifier (what represents). Graham Hough writes, for example: "In discussing whatever it is we mean by style we assume mat the primary choice, the choice of subject-matter in the large sense, has already been made.... We can talk about the secondary choice, the stylistic one, in various ways.... But whatever point of view we adopt, it will be the verbal ordonnance that we discuss, not the oudines of die mym, the facts of the case, the ideological or biographical substructure."2 Yet, as I have more than suggested, any what has (or can have) a corresponding how. Besides, both 59 60Philosophy and Literature in formal criticism and informal exchanges, we frequently seem to refer to the how of a signified. When we say, for instance, that a given short story is Kafkaesque, we do not, I think, have its verbal style in mind; and when we claim that a novelistic episode or character comes right out of Alice in Wonderland, we are not talking about linguistic matters. Of course, the fact mat we seem to refer to the how of signifieds (or that we do refer to the how of signifiers) does not necessarily mean mat mere are such things (or that we can describe them). But I suspect that there are (and that we can). I would like to discuss in what follows some of the ways in which we might be able to talk of the style of represented "worlds" more or less meaningfully and systematically and some of the problems attending such talk. For the sake of convenience and because of personal affinities, I will restrict myself to verbal narratives. I will not make a distinction between literary and non-literary ones, since such a distinction is formally and even pragmatically quite problematic. IfI concentrate on fictional narratives, it is, again, because of personal affinities and not because I mink that, when it comes to style, (ontological or other) differences between fiction and nonfiction make very much of a difference. As I will argue, style is in the eye ofthe beholder and, from that point of view, whether or not something is compossible with one's knowledge of the "real world" does not necessarily make it function as fictional or nonfictional; indeed, I am constandy surprised by the real world and I could easily recount... (shrink)

Saul Kripke did more than anyone else to bring possible worlds into the contemporary philosophical discourse, first with his more formal work on the model theory for modal logic in the 1960s, and then with his more philosophical lectures on reference and modality, delivered in January 1970, that used the possible worlds apparatus informally to clarify the relations between semantic issues about names and metaphysical issues about individuals and kinds. Possible worlds semantics have been widely applied since then, (...) both in philosophy and in other fields such as linguistic semantics and pragmatics, theoretical computer science, and game theory. Kripke’s work, along with that of David Lewis, stimulated an ongoing debate about the nature and metaphysical status of possible worlds. Kripke himself has had little to say about the issues raised in this debate, in print, beyond what he said in Naming and Necessity and in brief remarks in a preface to a later edition of the lectures, published in 1980. But there is a clear view of the nature of possible worlds, and of the status of an explanation of modality in terms of possible worlds, implicit in the lectures and the preface. -/- The central focus of the post–Naming and Necessity debate about possible worlds has been a contrast between David Lewis’s modal realism and various versions of actualism. On this general issue, it is clear enough where Kripke stands: His criticisms of what he describes as the “other countries” picture of possible worlds are an explicit rejection of Lewis’s realism about possible worlds. But the rejection of modal realism raises a range of further questions: What exactly are possible worlds (or possible states of the world, which Kripke suggests would be less misleading terminology)? What contribution do they make to the explanation of modal discourse, and of the distinctive facts that modal discourse is used to state? Does the slogan “necessity is truth in all possible worlds” provide, or point to, a reductive analysis of necessity? Are possible worlds, in some sense, prior to modal operators and modal auxiliaries? If not, in what sense are they explanatory? How are possible worlds, or counterfactual situations, specified? How do they contribute to our understanding of specific meta- physical questions about the relations between particular individuals and their qualitative characteristics, the kinds to which they belong, and the matter of which they are constituted? How are we to understand the possible existence of individuals that do not actually exist? What is the status of the domains of individuals in merely possible worlds? -/- My aim in this chapter is to spell out the views about some of these foundational questions that are expressed or at least implicit in Kripke’s lectures. In Section 2, I discuss the general contrast between modal realism and actualism and questions about the kind of explanation that possible worlds provide for modal discourse and modal facts. In Section 3, I look at Kripke’s views about how possible worlds are specified, in particular at the role of individuals in specifying possible worlds. In Section 4, I consider the problems about merely possible individuals. (shrink)

The beliefs of the agents in a multi-agent system have been formally modelled in the last decades using doxastic logics. The possible worlds model and its associated Kripke semantics provide an intuitive semantics for these logics, but they commit us to model agents that are logically omniscient. We propose a way of avoiding this problem, using a new kind of entities called subjective situations. We define a new doxastic logic based on these entities and we show how the (...) belief operators have some desirable properties, while avoiding logical omniscience. A comparison with two well-known proposals (Levesque's logic of explicit and implicit beliefs and Thijsse's hybrid sieve systems) is also provided. (shrink)

In relation to semantics, “grounding” has two relevant meanings. “Symbol grounding” is the process of connecting symbols to perception and the world. “Communicative grounding” is the process of interactively adding to common ground in dialog. Strategies for grounding in human communication include, crucially, strategies for resolving troubles caused by various kinds of miscommunication. As it happens, these two processes of grounding are closely related. As a side-effect of grounding an utterance, dialog participants may adjust the meanings they assign to (...) linguistic expressions, in a process of semantic coordination. Meanings of at least some expressions include perceptual aspects which enable DPs to classify entities as falling under the expression or not based on their perception of those entities. We show how perceptual grounding of symbols can be achieved in a process of interactively adding to common ground. This requires that perceptual aspects of meaning can be updated as a result of participating in linguistic interaction, thereby enabling fine-grained semantic coordination of perceptually grounded linguistic meanings. A formal semantics for low-level perceptual aspects of meaning is presented, tying these together with the logical-inferential aspects of meaning traditionally studied in formal semantics. The key idea is to model perceptual meanings as classifiers of perceptual input. This requires a framework where intensions are represented independently of extensions, and structured objects which can be modified as a result of learning. We use Type Theory with Records, a formal semantics framework which starts from the idea that information and meaning are founded on our ability to perceive and classify the world, that is, to perceive objects and situations as being of types. As an example of our approach, we show how a simple classifier of spatial information based on the Perceptron can be cast in TTR. (shrink)

When do two sentences say the same thing, that is, express the same content? We defend two-component (2C) semantics: the view that propositional contents comprise (at least) two irreducibly distinct constituents, (1) truth-conditions, and (2) subject-matter. We contrast 2C with one-component (1C) semantics, focusing on the view that subject-matter is reducible to truth- conditions. We identify exponents of this view and argue in favor of 2C. An appendix proposes a general formal template for propositional 2C semantics.

In the history of formal semantics, the successful joining of linguistic and philosophical work brought with it some difficult foundational questions concerning the nature of meaning and the nature of knowledge of language in the domain of semantics: questions in part about “what’s in the head” of a competent language-user. This paper, part of a project on the history of formal semantics, revisits the central issues of (Partee, 1979) in a historical context, as a clash (...) between two traditions, Fregean and Chomskyan, a clash that accompanied early work combining Montague’s semantics with Chomskyan syntax. Recent advances in philosophy of mind (from, e.g., Stalnaker and Burge) go a long way towards changing the framework of arguments about “psychological reality” and “competence”, challenging the suppositions on which the original dichotomy rested, thus largely defusing the tension. (shrink)

This 4000 word essay was selected for a special issue of 'Image & Narrative' (Issue 18, September 2007), on 'Thinking Pictures', guest edited by Hanneke Grootenboer, author of 'The Rhetoric of Perspective' (University of Chicago Press, 2005). 'Image & Narrative' is a peer-reviewed e-journal on visual narratology, with essays reviewed by at least two members of the editorial board. The essay addresses contemporary arguments on spectatorship within the philosophy of art. It examines different ways by which internal and external (...) spectators negotiate painting's two perspectives: the absent scene represented by the picture's marked surface and the external point of view of a beholder, standing in real space. The essay argues that if the imaginative engagement provided by an internal spectator (see Wollheim) renders the picture plane transparent, so that we might enter the virtual space of the painting, then in paintings integrated into their architectural contexts the surface plays a semantic role in both connecting and separating different realms. If the picture surface disappears for the internal spectator, then for works integrated into their architectural settings its status is heightened and negated as a threshold separating spatiotemporal realms. The essay challenges semiotic accounts of painting. While linguistic derived semiotic accounts negate a spatial engagement, in Hubert Damisch's proposal that painting is a form of thinking, structure emerges from within the loop of the perspective paradigm. The essay argues, however, that if certain paintings 'think', they do so only in the sense that thought permeates a phenomenological whole where perception and imagination merge. The essay is a development of 'A spatiality of situation', published by Art In-sight, Filmwaves magazine (no. 31, 2006), guest edited with Adam Kossoff. A related paper, 'Levels of Reality', was given at the 'Real Things: matter, materiality, representation' conference at the University of York, 5-8 July 2007. (shrink)

The beginning of this century hailed a new paradigm in linguistics, the paradigm brought about by de Saussure's Cours de Linguistique Generale and subsequently elaborated by Jakobson, Hjelmslev and other linguists. It seemed that the linguistics of this century was destined to be structuralistic. However, half of the century later a brand new paradigm was introduced by Chomsky's Syntactic Structures followed by Montague's formalization of semantics. This new turn has brought linguistics surprisingly close to mathematics and logic, and has (...) facilitated a direct practical exploitation of linguistic theory by computer science. (shrink)

The grammatical paradigm used to be a model for entire areas of cognitive science. Its primary tenet was that theories are axiomatic-like systems. A secondary tenet was that their predictions should be tested quickly and in great detail with introspective judgments. While the grammatical paradigm now often seems passé, we argue that in fact it continues to be as efficient as ever. Formal models are essential because they are explicit, highly predictive, and typically modular. They make numerous critical predictions, (...) which must be tested efficiently; introspective judgments do just this. We further argue that the grammatical paradigm continues to be fruitful. Within linguistics, implicature theory is a recent example, with a combination of formal explicitness, modularity, and interaction with experimental work. Beyond traditional linguistics, the grammatical paradigm has proven fruitful in the study of gestures and emojis; literature (“Free Indirect Discourse”); picture semantics and comics; music and dance cognition; and even reasoning and concepts. We argue, however, that the grammatical paradigm must be adapted to contemporary cognitive science. Computational methods are essential to derive quantitative predictions from formal models (Bayesian pragmatics is an example). And data collection techniques offer an ever richer continuum of options, from introspective judgments to large-scale experiments, which makes it possible to optimize the cost/benefit ratio of the empirical methods that are chosen to test theories. (shrink)

Confucius became a subject for visual representation after the Han court formally endorsed his teachings, and his earliest images appeared in schools and offering shrines. As his official cult evolved, and until the 1530 ritual reform, iconic portraits of Confucius and his disciples received offerings in temples throughout China. During the Song period, his portrayals became more diverse, and some reproduced pictures kept by his Kong descendants in Qufu曲阜and Quzhou衢州. Attributions to the Tang painter Wu Daozi 吳道子became customary and (...) persistent, despite tenuous connections. Until the 20th century, images of Confucius primarily concerned the literate elite, for whom he was an exemplar of moral cultivation. Pictorial biographies of Confucius, made in considerable numbers from the middle Ming onward, brought him more vividly to life and to a wider range of society. In recent decades, new images of Confucius have evolved to serve a range of contemporary purposes. (shrink)

Situation theory is the result of an interdisciplinary effort to create a full-fledged theory of information. Created by scholars and scientists from cognitive science, computer science, AI, linguistics, logic, philosophy, and mathematics, the theory is forging a common set of tools for the analysis of phenomena from all these fields. This volume presents work that evolved out of the Second Conference on Situation Theory and its Applications. Twenty-six essays exhibit the wide range of the theory, covering such topics (...) as natural language semantics, philosophical issues about information, mathematical applications, and the visual representation of information in computer systems.Jon Barwise is a professor of philosophy, mathematics, and logic at Indiana University in Bloomington. Jean Mark Gawron is a researcher at SRI International and a consultant at Hewlett-Packard Laboratories. Gordon Plotkin is a professor of theoretical computer science at the University of Edinburgh. Syun Tutiya is in the philosophy department at Chiba University in Japan. (shrink)

The paper analyzes the visual aspect of metaphors, offering a new theory of metaphor that characterizes its syntactic structure, material composition and visuality as its essence. It will accordingly present the metaphorical creating or transfiguring, as well as conceiving or understanding, of one thing as a different one, as a visual ability. It is a predication by means of producing non-conventional compositions – i.e., by compositional, or even aesthetic, means. This definition is aimed to apply to the various (...) kinds of metaphors: conceptual, linguistic, visual, and material. It will thus challenge the definition of metaphor as a conceptual or linguistic phenomenon in nature that is based on its semantic mechanism, broad concepts, and cognitive value. Those definitions have been prevalent since the second half of 20th century, under the influence of the philosophy of language, and later of cognitive studies. (shrink)

(1) This is Part 2 of the semantic theory I call TM. In Part 1, I developed TM as a theory in the analytic philosophy of language, in lexical semantics, and in the sociology of relating occasions of statement production and comprehension to formal and informal lexicographic conclusions about statements and lexical items – roughly, as showing how synchronic semantics is a sociological derivative of diachronic, person-relative acts of linguistic behavior. I included descriptions of new cognitive psychology (...) experimental paradigms which would allow us to precisely measure the two constituents of semantics – meaning and reference – both at the level of individual speech acts and at the level of societal convergences, i.e. at both the token and type levels. -/- (2) In the Introduction, I recapitulate the arguments of Part 1. The Introduction also develops some analytic philosophical and lexical semantics themes not discussed in Part 1. -/- (3) After the Introduction, I present neural TM (nTM) as a theory of the neural mechanisms and processes which give rise to these person/occasion-relative acts of linguistic behavior. I develop nTM at three levels, the first two of which describe linguistic/semantic functions independently of their cortical locations. At the first level, I describe individual word-to-word and word-to-object connections. At the second level, I describe the corresponding structuralist networks of which they are the individual components. At this level, I introduce some key linguistic concepts of TM – its graded meaning, reference, and generalization sets, and the types of statements which express various levels of word-to-word and word-to-object relationships among lexical items which, because of the constraints they impose on the use of those lexical items in statements we produce and comprehend, are concepts. This constitutes the second structural level of nTM. -/- (4) At the third level, I associate the non-localized structures of the previous levels with cortically located neural structures and with the fasciculi that connect them. I distinguish neural areas in which primary (phonetic) and secondary (orthographic) lexicons are stored in long-term memory. I also describe the embodied concepts which co-exist in the anterior temporal lobes with the images they lexicalize. These concepts are often said to name physical objects and their features, although what they in fact name are kinds of physical objects and features. I describe how conceptual constraints and referential constraints interact to channel our intentions to say how things are into statements which are semantically well-formed, and which consequently successfully communicate information. -/- (5) Following this presentation of nTM, I examine five prominent neural semantic theories. I point out what is wrong with each of them as far as their explanations of semantics are concerned, and I also indicate how nTM can replace the “semantic cores” of those theories. -/- (6) The two basic mistakes made by neuroscience semantic theories, as I will explain, are (i) that all but one of them regard semantics as a matter of the association of words with perceptual images, and of generalizations from those associations; and (ii) that they all rely on an unspecified set of neural structures which purportedly encode the meaning of concepts in abstraction from their phonological and orthographic forms. nTM maintains, in contrast, that there are no abstract neural representations of semantic content. Neural constraints on our linguistic behavior, especially on our ascriptive and co-ascriptive use of words, express the semantic constraints on those words which make them concepts. That is the semantic content of words. -/- (7) I next consider several results from neuroscience experimental data which have been given one interpretation by one or another of the standard neurosemantic theories, but to which nTM gives a different interpretation. I include several predictions which I have found neither confirmed nor disconfirmed in the experimental neuroscience literature. -/- (8) After a concluding section in which I summarize the major changes to neurosemantic theory introduced by TM, and the analytic philosophy of language and lexical semantics contexts within which TM is situated, there follows an appendix in which I discuss neural net AI, and make some recommendations for implementing nTM in silicon. (shrink)

In the formal semantics based on modern type theories, common nouns are interpreted as types, rather than as predicates of entities as in Montague’s semantics. This brings about important advantages in linguistic interpretations but also leads to a limitation of expressive power because there are fewer operations on types as compared with those on predicates. The theory of coercive subtyping adequately extends the modern type theories and, as shown in this paper, plays a very useful role in (...) making type theories more expressive for formal semantics. It not only gives a satisfactory solution to the basic problem of ‘multiple categorisation’ caused by interpreting common nouns as types, but provides a powerful formal framework to model interesting linguistic phenomena such as copredication, whose formal treatment has been found difficult in a Montagovian setting. In particular, we show how to formally introduce dot-types in a type theory with coercive subtyping and study some type-theoretic constructs that provide useful representational tools for reference transfers and multiple word meanings in formal lexical semantics. (shrink)

This thesis challenges a general prejudice against visualization in the history of logic and mathematics, by providing a semantic analysis of two graphical representation systems--a traditional Venn diagram representation system and an extension of it. While Venn diagrams have been used to solve problems in set theory and to test the validity of syllogisms in logic, they have not been considered valid proofs but heuristic tools for finding valid formal proofs. ;I present Venn diagrams which have been used in (...) logic as a formal system of representation equipped with its own syntax and semantics. I also show that Venn-I, with the rules of transformation that I specify, is sound and complete. The soundness of Venn-I refutes a worry that mathematicians and logicians have about graphical representation: Fallacies often have arisen from the misuse of diagrams in mathematical proofs, especially in geometry. However, the validity of the transformation rules assures us that the correct application of these rules will not lead to fallacies. ;I extend Venn-I so that this new system has a general way to convey disjunctive information. A new formation rule is introduced and new transformation rules are added. I call this extended system Venn-II. The semantics of Venn-II is formalized as an extension of the semantics of Venn-I. The soundness and completeness of Venn-II are also proven. I then show that Venn-II and a monadic first-order language are equivalent in their expressiveness. ;The analysis of Venn-I and Venn-II leads to interesting issues which have their analogues in other deductive systems. I conclude the thesis with a discussion of some fundamental differences between graphical systems and linguistic systems. (shrink)

Experimental research has shown that pairs of stimuli which are congruent and assumed to 'go together' are recalled more effectively than an item presented in isolation. Will this multisensory memory benefit occur when stimuli are richer and longer, in an ecological setting? In the present study, we focused on an everyday situation of audio-visual learning and manipulated the relationship between audio guide tracks and viewed portraits in the galleries of the Tate Britain. By varying the gender and narrative (...) style of the voice-over, we examined how the perceived congruency and assumed unity of the audio guide track with painted portraits affected subsequent recall. We show that tracks perceived as best matching the viewed portraits led to greater recall of both sensory and linguistic content. We provide the first evidence that manipulating crossmodal congruence and unity assumptions can effectively impact memory in a multisensory ecological setting, even in the absence of precise temporal alignment between sensory cues. (shrink)

The beginning of this century hailed a new paradigm in linguistics, the paradigm brought about by de Saussure's Cours de Linguistique Genérále and subsequently elaborated by Jakobson, Hjelmslev and other linguists. It seemed that the linguistics of this century was destined to be structuralistic. However, half of the century later a brand new paradigm was introduced by Chomsky's Syntactic Structures followed by Montague's formalization of semantics. This new turn has brought linguistics surprisingly close to mathematics and logic, and has (...) facilitated a direct practical exploitation of linguistic theory by computer science. (shrink)

In this paper, I utilise the growing literature on scientific modelling to investigate the nature of formal semantics from the perspective of the philosophy of science. Specifically, I incorporate the inferential framework proposed by Bueno and Colyvan : 345–374, 2011) in the philosophy of applied mathematics to offer an account of how formal semantics explains and models its data. This view produces a picture of formal semantic models as involving an embedded process of inference and (...) representation applying indirectly to linguistic phenomena. The final aim of the paper is directed at proposing a novel account of the syntax–semantics interface while shedding light on empty categories, semantically null forms, underspecified content and compositionality as a whole. (shrink)

Situation semantics was developed as an alternative to possible worlds semantics. In situation semantics, linguistic expressions are evaluated with respect to partial, rather than complete, worlds. There is no consensus about what situations are, just as there is no consensus about what possible worlds or events are. According to some, situations are structured entities consisting of relations and individuals standing in those relations. According to others, situations are particulars. In spite of unresolved foundational issues, the (...) partiality provided by situation semantics has led to some genuinely new approaches to a variety of phenomena in natural language semantics. In the way of illustration, this article includes relatively detailed overviews of a few selected areas where situation semantics has been successful: implicit quantifier domain restrictions, donkey pronouns, and exhaustive interpretations. It moreover addresses the question of how Davidsonian event semantics can be embedded in a semantics based on situations. Other areas where a situation semantics perspective has led to progress include attitude ascriptions, questions, tense, aspect, nominalizations, implicit arguments, point of view, counterfactual conditionals, and discourse relations. (shrink)

Formal semantics is an approach to SEMANTICS1, the study of meaning, with roots in logic, the philosophy of language, and linguistics, and since the 1980’s a core area of linguistic theory. Characteristics of formal semantics to be treated in this article include the following: Formal semanticists treat meaning as mind-independent (though abstract), contrasting with the view of meanings as concepts “in the head” (see I-LANGUAGE AND E-LANGUAGE and MEANING EXTERNALISM AND INTERNALISM); formal semanticists distinguish (...) semantics from knowledge of semantics (Lewis 1975, Cresswell 1978), which has consequences for the notion of semantic COMPETENCE. A central part of the meaning of a sentence on this approach is its TRUTH CONDITIONS, and most although not all formal semantics is model-theoretic, relating linguistic expressions to model-theoretically constructed semantic values cast in terms of truth, REFERENCE, and possible worlds. This sets formal semantics apart from approaches which view semantics as relating a sentence just to a representation on another linguistic “level” (LOGICAL FORM) or a representation in an innate LANGUAGE OF THOUGHT. The formal semanticist could accept such representations as an aspect of semantics but would insist on asking what the model-theoretic semantic interpretation of the given representationlanguage is (Lewis 1970). Formal semantics is centrally concerned with COMPOSITIONALITY at the SYNTAX-SEMANTICS INTERFACE, how the meanings of larger constituents are built up from the meanings of their parts on the basis of their syntactic structure, and with the relation between compositional SENTENCE MEANING and meaning in discourse. (shrink)

Formal semantics and pragmatics as they have developed since the late 1960's have been shaped by fruitful interdisciplinary collaboration among linguists, philosophers, and logicians, among others, and in turn have had noticeable effects on developments in syntax, philosophy of language, computational linguistics, and cognitive science.In this paper I describe the environment in which formal semantics was born and took root, highlighting the differences in ways of thinking about natural language semantics in linguistics and in philosophy (...) and logic. With Montague as a central but not solo player in the story, I reflect on crucial developments in the 1960's and 70's in linguistics and philosophy, and the growth of formal semantics and formal pragmatics from there. I discuss innovations, key players, and leading ideas that shaped the development of formal semantics and its relation to syntax, to pragmatics, and to the philosophy of language in its early years, and some central aspects of its early impact on those fields. (shrink)

Linguistic Semantics: An Introduction is the successor to Sir John Lyons's important textbook Language, Meaning and Context (1981).While preserving the general structure of the earlier book, the author has substantially expanded its scope to introduce several topics that were not previously discussed, and to take account of new developments in linguistic semantics over the past decade. The resulting work is an invaluable guide to the subject, offering clarifications of its specialised terms and explaining its relationship to formal (...) and philosophical semantics and to contemporary pragmatics. With its clear and accessible style it will appeal to a wide student readership. Sir John Lyons is one of the most important and internationally renowned contributors to the study of linguistics. His many publications include Introduction to Theoretical Linguistics (1968) and Semantics (1977). (shrink)

This volume is a comprehensive, up-to-date, and readable introduction to linguistic meaning. While partial to conceptual and typological approaches, the book also presents results from formal approaches. Throughout, the focus is on grammatical meaning -- the way languages delineate universal semantic space and encode it in grammatical form. Subjects covered by the author include: the domain of linguistic semantics and the basic tools, assumptions, and issues of semantic analysis; semantic properties of entities, events, and thematic roles; language and (...) space; tense, aspect, and the internal structure and temporal ordering of events; modality, negation, and the epistemology of the speaker; and modification and attribution. In contrast to most current treatments of semantics, this book is accessible to the beginning student of semantics and linguistics and also useful to the advanced practitioner. A textbook and reference work in a single volume, it can be used in a number of disciplines: psychology, computer science, and anthropology as well as linguistics. (shrink)

Formal semantics - the scientific study of meaning in natural language - is one of the most fundamental and long-established areas of linguistics. This Handbook offers a comprehensive, yet compact guide to the field, bringing together research from a wide range of world-leading experts. Chapters include coverage of the historical context and foundation of contemporary formal semantics, a survey of the variety of formal/logical approaches to linguistic meaning and an overview of the major areas of (...) research within current semantic theory, broadly conceived. The Handbook also explores the interfaces between semantics and neighbouring disciplines, including research in cognition and computation. This work will be essential reading for students and researchers working in linguistics, philosophy, psychology and computer science. (shrink)

This accessible introduction to formal, and especially Montague, semantics within a linguistic framework, presupposes no previous background in logic, but takes students step-by-step from simple predicate/argument structures and their interpretation to Montague's intentional logic.

This paper aims to argue for two related statements: first, that formal semantics should not be conceived of as interpreting natural language expressions in a single model (a very large one representing the world as a whole, or something like that) but as interpreting them in many different models (formal counterparts, say, of little fragments of reality); second, that accepting such a conception of formal semantics yields a better comprehension of the relation between semantics (...) and pragmatics and of the role to be played by formal semantics in the general enterprise of understanding meaning. For this purpose, three kinds of arguments are given: firstly, empirical arguments showing that the many models approach is the most straightforward and natural way of giving a formal counterpart to natural language sentences. Secondly, logical arguments proving the logical impossibility of a single universal model. And thirdly, theoretical arguments to the effect that such a conception of formal semantics fits in a natural and fruitful way with pragmatic theories and facts. In passing, this conception will be shown to cast some new light on the old problems raised by liar and sorites paradoxes. (shrink)

The thesis is an attempt to give a precise formal semantics for various time-referential linguistic categories of English such as tense, perfect, progressive and Aktionsart or "action type", with the ultimate goal of explaining why with a verb phrase such as walk the inference from, for instance, John is walking to John has walked is intuitively valid, while with a verb phrase such as build a house the even weaker inference from John is building a house to John (...) will have built a house is intuitively invalid. The difference is seen as one of the Aktionsart of the sentences involved. ;Accordingly, a theory of Aktionsarten is developed in chapter 2. Four different Aktionsarten are formally distinguished. In chapter 3 a theory of the tenses and their interaction with temporal adverbials is worked out. Chapter 4 is about the perfect. The so-called "extended now" theory of the perfect is given a precise treatment. Finally, chapter 5 deals with some problems of the English progressive. Especially the so-called "imperfective paradox" is given a solution. ;The method I use is that of model-theoretic semantics applied to natural languages, as it was first developed by the logician, Richard Montague. (shrink)

In formal semantics, structure is treated as the essential ingredient in the creation of sentence meaning from individual word meaning. This book introduces some of the foundational concepts, principles and techniques in the formal semantics of natural language and outlines the mathematical principles that underlie linguistics meaning. Using English examples, Yoad Winter presents the most useful tools and concepts of formal semantics in an accessible style and includes a variety of practical exercises so that (...) readers can learn to utilize these tools effectively. For readers with an elementary background in set theory and linguistics or with an interest in mathematical modelling, this fascinating study is an ideal introduction to natural language semantics. Designed as a quick yet thorough introduction to one of the most vibrant areas of research in modern linguistics today this volume reveals the beauty and elegance of the mathematical study of meaning. (shrink)

Barsalou's contribution converges with basic ideas and empirical findings of cognitive linguistics. They posit the same general architecture. The perceptual grounding of conceptual structure is a central tenet of cognitive linguistics. Our capacity to construe the same situation in alternate ways is fundamental to cognitive semantics, and numerous parallels are discernible between conceptual construal and visual perception. Grammar is meaningful, consisting of schematized patterns for the pairing of semantic and phonological structures. The meanings of grammatical elements reside (...) primarily in the construal they impose on conceptual content. This view of linguistic structure appears to be compatible with Barsalou's proposals. (shrink)

I will try to do three things in this paper. First, I want to situate certain problems in natural language semantics with respect to larger trends in logicism, including: (i) Attempts by positivist philosophers earlier in this century to provide a logical basis for the physical sciences; (ii) Attempts by linguists and logicians to develop a “natural language ontology” (and, presumably, a logical language that is related to this ontology by formally explicit rules) that would serve as a framework (...) for natural language semantics. (shrink)

Like Spanish moss on a live oak tree, the scientific study of meaning in language has expanded in the last 100 years, and continues to expand steadily. In this essay I want to chart some central themes in that expansion, including their histories and their important figures. Our attention will be directed toward what is called 'formal semantics', which is the adaptation to natural language of analytical techniques from logic.[1] The first, background, section of the paper will survey (...) the changing attitudes of linguists toward semantics into the last third of the century. The second and third sections will examine current formal approaches to meaning. In the final section I will summarize some of the common assumptions of the approaches examined in the middle sections of the paper, sketch a few alternatives, and make some daring predictions. (shrink)

Linguists often sharply distinguish the different modules that support linguistics competence, e.g., syntax, semantics, pragmatics. However, recent work has identified phenomena in syntax (polarity sensitivity) and pragmatics (implicatures), which seem to rely on semantic properties (monotonicity). We propose to investigate these phenomena and their connections as a window into the modularity of our linguistic knowledge. We conducted a series of experiments to gather the relevant syntactic, semantic and pragmatic judgments within a single paradigm. The comparison between these quantitative data (...) leads us to four main results, (i) Our results support a departure from one element of the classical Gricean approach, thus helping to clarify and settle an empirical debate. This first outcome also confirms the soundness of the methodology, as the results align with standard contemporary accounts of scalar implicature (SI), (ii) We confirm that the formal semantic notion of monotonicity underlies negative polarity item (NPI) syntactic acceptability, but (iii) our results indicate that the notion needed is perceived monotonicity. We see results (ii) and (iii) as the main contribution of this study: (ii) provides an empirical interpretation and confirmation of one of the insights of the model-theoretic approach to semantics, while (iii) calls for an incremental, cognitive implementation of the current generalizations, (iv) Finally, our results do not indicate that the relationship between NPI acceptability and monotonicity is mediated by pragmatic features related to Sis: this tells against elegant attempts to unify polarity sensitivity and Sis (pioneered by Krifka and Chierchia). These results illustrate a new methodology for integrating theoretically rigorous work in formal semantics with an experimentally-grounded cognitively-oriented view of linguistic competence. (shrink)