Computational Linguistics

The success of deep learning in natural language processing raises intriguing questions about the nature of linguistic meaning and ways in which it can be processed by natural and artificial systems. One such question has to do with subword segmentation algorithms widely employed in language modeling, machine translation, and other tasks since 2016. These algorithms often cut words into semantically opaque pieces, such as ‘period’, ‘on’, ‘t’, and ‘ist’ in ‘period|on|t|ist’. The system then represents the resulting segments in a dense (...) vector space, which is expected to model grammatical relations among them. This representation may in turn be used to map ‘period|on|t|ist’ (English) to ‘par|od|ont|iste’ (French). Thus, instead of being modeled at the lexical level, translation is reformulated more generally as the task of learning the best bilingual mapping between the sequences of subword segments of two languages; and sometimes even between pure character sequences: ‘p|e|r|i|o|d|o|n|t|i|s|t’ → ‘p|a|r|o|d|o|n|t|i|s|t|e’. Such subword segmentations and alignments are at work in highly efficient end-to-end machine translation systems, despite their allegedly opaque nature. The computational value of such processes is unquestionable. But do they have any linguistic or philosophical plausibility? I attempt to cast light on this question by reviewing the relevant details of the subword segmentation algorithms and by relating them to important philosophical and linguistic debates, in the spirit of making artificial intelligence more transparent and explainable. (shrink)

Objective. Study of the validity and reliability of the discourse approach for the psycholinguistic understanding of the nature, structure, and features of the linguistic consciousness functioning. -/- Materials & Methods. This paper analyzes artificial neural network models built on the corpus of texts, which were obtained in the process of experimental research of the coronavirus quarantine concept as a new category of linguistic consciousness. The methodology of feedforward artificial neural networks (multilayer perceptron) was used in order to assess the possibility (...) of predicting the leading texts semantics based on the discourses ranks and their place in the respective linear sequence. Same baseline parameters were used to predict respondents' self-assessments of changes in their psychological well-being and in daily life routine during the quarantine, as well as to predict their preferences of the quarantine strategies. The study relied on basic ideas about discourse as a meaning constituted by the dispersion of other meanings (Foucault). The same dispersion mechanism realizes itself in interdiscourse interaction, forming a discursive formation at a higher level. The method of T-units (Hunt) was used to identify and count discourses in the texts. The ranking of discourses was provided based on the criterion of their semantic-syntactic autonomy. -/- Results. The conducted neural network modeling revealed a high accuracy in predicting the work of the linguistic consciousness functions associated with retrospective self-assessment and anticipatory imagination of the respondents. Another result of this modeling is a partial confirmation of the assumption concerning existence a relationship between the structural parameters of the discursive field (the rank of the discourses and their place in the respective linear sequence) and the leading semantics of the text. -/- Conclusions. A discourse approach to the study of linguistic consciousness, understanding of its structure and functioning features seems to be reasonably appropriate. The implementation of the approach presupposes the need to form a base of linguistic corpora with the inclusion in each text markup of such parameters as: the presence of specific discourses, their ranks, positions in the linear sequence of discourses. (shrink)

The manuscript presents a summary of the results of the linguistic semantics study of Covid-19 related quarantine. Research conducted on a sample of Russian speaking Ukrainians. Found content and structure of the respective discursive field. Described features of inter-discourse connections. Established that the actualization of some discourses is accompanied by the deactivation of others, what makes quarantine semantics biased. Also, it was suggested that some of the discourses are indirectly positively associated and form the semantic core of the quarantine concept.

Objective. Study of the Covid-19 related quarantine concept as an emerging category of linguistic consciousness of Ukrainians. -/- Materials & Methods. The strategy of the study is based on the logical and methodological concept of inductivism. Respondents were asked to write down their own understanding of the quarantine, formulate an appropriate definition and describe the situation, which in their opinion is the exact opposite to quarantine. Respondents also assessed how much their psychological well-being, their daily lifestyle during quarantine had changed, (...) and ranked their preferences for the quarantine strategies proposed to them. A discursive analysis was applied to the obtained texts, as a result of which nine discourses were identified. These data, along with some socio-demographic characteristics were subjected to multidimensional mathematical and statistical processing. -/- Results. Covid-19 related quarantine is represented in the linguistic consciousness of Russian-speaking Ukrainians by a discursive field, which includes at least nine recognizable, semantically autonomous discourses. Empirically discovered such phenomenon as – inter-discourse semantic dissociation. Its essence is a statistically significant reduction in the probability of some discourses appearing in the texts when others are being actualized there. This feature is associated with the innate negativity of the language and determines the semantic biasing of the quarantine concept. Inter-discourse semantic dissociation, as well as the influence of non-discursive factors constitute the discursive formation of the quarantine concept, which is qualitatively characterized by the hierarchical relationship of its components. At the same time, it was revealed that part of the discourses interacting “horizontally” are indirectly associated “vertically”, which forms the stable semantic core of the quarantine concept. Partial empirical confirmation has been found of the previously put forward assumptions about: a) the existence of such a relationship between discourses and psychological defense mechanisms (as character-forming factors) that contributes to the hierarchical structure construction of discursive formations; b) the differential nature of discourses and mechanisms of psychological defenses interaction, which makes it possible to single out the discursive aspect in a characterological profile and consider a discourse as an operator of characterological semantics. -/- Conclusions. An empirical study made it possible to form a primary idea of the substantive and structural semantic features of the quarantine concept, as an emerging category of linguistic consciousness of Ukrainians. The results obtained have a potentially useful application perspective. Thus, when implementing anti-epidemiological measures, it is important to consider the studied features of the discursive formation hierarchical structure of the quarantine concept, since discursive semantics are associated with cognitive focusing, which, in turn, affects the behavior direction and its upshots. (shrink)

In this chapter we use methods of corpus linguistics to investigate the ways in which mathematicians describe their work as explanatory in their research papers. We analyse use of the words explain/explanation (and various related words and expressions) in a large corpus of texts containing research papers in mathematics and in physical sciences, comparing this with their use in corpora of general, day-to-day English. We find that although mathematicians do use this family of words, such use is considerably less prevalent (...) in mathematics papers than in physics papers or in general English. Furthermore, we find that the proportion with which mathematicians use expressions related to ‘explaining why’ and ‘explaining how’ is significantly different to the equivalent proportion in physics and in general English. We discuss possible accounts for these differences. (shrink)

Recursion: A Computational Investigation into the Representation and Processing of Language. By Lobina David.

This article contains the results of a theoretical analysis of the phenomenon of natural language understanding (NLU), as a methodological problem. The combination of structural-ontological and informational-psychological approaches provided an opportunity to describe the subject matter field of NLU, as a composite function of the mind, which systemically combines the verbal and discursive structural layers. In particular, the idea of NLU is presented, on the one hand, as the relation between the discourse of a specific speech message and the meta-discourse (...) of a language, in turn, activated by the need-motivational factors. On the other hand, it is conceptualized as a process with a specific structure of information metabolism, the study of which implies the necessity to differentiate the affective (emotional) and need-motivational influences on the NLU, as well as to take into account their interaction. At the same time, the hypothesis about the influence of needs on NLU under the scenario similar to the pattern of Yerkes-Dodson is argued. And the theoretical conclusion that emotions fulfill the function of the operator of the structural features of the information metabolism of NLU is substantiated. Thus, depending on the modality of emotions in the process of NLU, it was proposed to distinguish two scenarios for the implementation of information metabolism - reduction and synthetic. The argument in favor of the conclusion about the productive and constitutive role of emotions in the process of NLU is also given. (shrink)

This study focuses on the interdisciplinary issue of language corpora mining for the purpose of cultural analytics. The authors utilize the comparative cross -cultural approach to highlight the dynamics of “revolution” concept throughout two and a half centuries. They study concept application through the word occurrence in five languages by the means of Google Books Ngram Viewer and trace the waves of concept “popularity”. Most of the word application peaks can be corresponded to the political events called “revolutions” but there (...) is also an exception. (shrink)

Psychologists have used experimental methods to study language for more than a century. However, only with the recent availability of large-scale linguistic databases has a more complete picture begun to emerge of how language is actually used, and what information is available as input to language acquisition. Analyses of such “big data” have resulted in reappraisals of key assumptions about the nature of language. As an example, we focus on corpus-based research that has shed new light on the arbitrariness of (...) the sign: the longstanding assumption that the relationship between the sound of a word and its meaning is arbitrary. The results reveal a systematic relationship between the sound of a word and its meaning, which is stronger for early acquired words. Moreover, the analyses further uncover a systematic relationship between words and their lexical categories—nouns and verbs sound differently from each other—affecting how we learn new words and use them in sentences. Together, these results point to a division of labor between arbitrariness and systematicity in sound-meaning mappings. We conclude by arguing in favor of including “big data” analyses into the language scientist's methodological toolbox. (shrink)

This book takes concepts developed by researchers in theoretical computer science and adapts and applies them to the study of natural language meaning. Summarizing over a decade of research, Chris Barker and Chung-chieh Shan put forward the Continuation Hypothesis: that the meaning of a natural language expression can depend on its own continuation.

In this paper we propose a way to deal with natural language inference by implementing Modern Type Theoretical Semantics in the proof assistant Coq. The paper is a first attempt to deal with NLI and natural language reasoning in general by using the proof assistant technology. Valid NLIs are treated as theorems and as such the adequacy of our account is tested by trying to prove them. We use Luo’s Modern Type Theory with coercive subtyping as the formal language into (...) which we translate natural language semantics, and we further implement these semantics in the Coq proof assistant. It is shown that the use of a MTT with an adequate subtyping mechanism can give us a number of promising results as regards NLI. Specifically, it is shown that a number of inference cases, i.e. quantifiers, adjectives, conjoined noun phrases and temporal reference among other things can be successfully dealt with. It is then shown, that even though Coq is an interactive and not an automated theorem prover, automation of all of the test examples is possible by introducing user-defined automated tactics. Lastly, the paper offers a number of innovative approaches to NL phenomena like adjectives, collective predication, comparatives and factive verbs among other things, contributing in this respect to the theoretical study of formal semantics using MTTs. (shrink)

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this (...) manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

Utterances can convey more information than they explicitly encode, and speakers exploit communicative conventions in order to say more with less. However, the burden this places on perceivers is not well understood. This chapter examines the effect of speaker-specific information on pragmatic inferences using data from an experiment which investigated the time course of the use of pragmatic information in language comprehension. Previous evidence suggests that comprehenders who encounter a referential form, including a modifier that commonly indicates contrastiveness, assume that (...) the referential form is being used contrastively. Thus, they consequently look to the relevant object in the scene very early when processing the referential expression. In this case, people appear to be implicitly following H. P. Grice’s maxim of quantity. This chapter demonstrates that comprehenders stop using contrastive information early in resolving reference when speakers consistently do not use contrastive elements appropriately, consistent with a pragmatic explanation of referential contrast effects. (shrink)

This paper investigates whether the model of local rhetorical coherence suggested in Knott et al. (2001) can boost the performance of the Centering-based metrics of entity coherence employed by Karamanis et al. (2004) for the task of information ordering. Rhetorical coherence is integrated into the way Centering’s basic data structures are derived from the annotated features of the GNOME corpus. The results indicate that (a) the simplest metric continues to perform better than its competitors even when local rhetorical coherence is (...) taken into account, and (b) this extra coherence constraint decreases its performance. (shrink)

Probabilistic methods are providing new explanatory approaches to fundamental cognitive science questions of how humans structure, process and acquire language. This review examines probabilistic models defined over traditional symbolic structures. Language comprehension and production involve probabilistic inference in such models; and acquisition involves choosing the best model, given innate constraints and linguistic and other input. Probabilistic models can account for the learning and processing of language, while maintaining the sophistication of symbolic models. A recent burgeoning of theoretical developments and online (...) corpus creation has enabled large models to be tested, revealing probabilistic constraints in processing, undermining acquisition arguments based on a perceived poverty of the stimulus, and suggesting fruitful links with probabilistic theories of categorization and ambiguity resolution in perception. (shrink)

This has been made available gratis by the publisher. -/- This piece gives the raison d'etre for the development of the converters mentioned in the title. Three reasons are given, one linguistic, one philosophical, and one practical. It is suggested that at least /two/ independent converters are needed. -/- This piece ties together the extended paper "Abstracts from Logical Form I/II," and the short piece providing the comprehensive theory alluded to in the abstract of that extended paper in "Pragmatics, Montague, (...) and 'Abstracts from Logical Form'" by motivating the entire project from beginning to end. (shrink)

We combine state-of-the-art techniques from computational linguisticsand theorem proving to build an engine for playing text adventures,computer games with which the player interacts purely through naturallanguage. The system employs a parser for dependency grammar and ageneration system based on TAG, and has components for resolving andgenerating referring expressions. Most of these modules make heavy useof inferences offered by a modern theorem prover for descriptionlogic. Our game engine solves some problems inherent in classical textadventures, and is an interesting test case for (...) the interactionbetween natural language processing and inference. (shrink)

We present an implementation of a discourse parsing system for alexicalized Tree-Adjoining Grammar for discourse, specifying the integrationof sentence and discourse level processing. Our system is based on theassumption that the compositional aspects of semantics at thediscourse level parallel those at the sentence level. This coupling isachieved by factoring away inferential semantics and anaphoric features ofdiscourse connectives. Computationally, this parallelism is achievedbecause both the sentence and discourse grammar are LTAG-based and the sameparser works at both levels. The approach to an (...) LTAG for discourse has beendeveloped by Webber and colleagues in some recent papers. Our system takes a discourseas input, parses the sentences individually, extracts the basic discourseconstituent units from the sentence derivations, and reparses the discoursewith reference to the discourse grammar while using the same parser usedat the sentence level. (shrink)

Phraseology has been acquiring increasing relevance in the fields of linguistics, lexicography and language teaching. This volume brings together data from a variety of sources to arrive at a better understanding and description of various types of 'phrases' and 'phraseological units' as used in 'real' language. The different approaches and objectives of the papers attest to the pervasiveness of phraseology in different languages, sub-languages and registers. The role of different types of corpora is highlighted and native speakers' expectations, culture-specific factors, (...) context-dependent choices and genre specifications are explored. (shrink)

In recent decades, the analysis of phraseology has made use of the exploration of large corpora as a source of quantitative information about language. This paper intends to present the main lines of work in progress based on this empirical approach to linguistic analysis. In particular, we focus our attention on some problems relating to the morpho-syntactic annotation of corpora. The CORIS/CODIS corpus of contemporary written Italian, developed at CILTA – University of Bologna (Rossini Favretti 2000; Rossini Favretti, Tamburini, De (...) Santis in press), is a synchronic 100-million-word corpus and is being lemmatised and annotated with part-of-speech (POS) tags, in order to increase the quantity of information and improve data retrieval procedures (Tamburini 2000). The aim of POS tagging is to assign each lexical unit to the appropriate word class. Usually the set of tags is pre-established by the linguist, who uses his/her competence to identify the different word classes. The very first experiments we made revealed how the traditional part-of-speech distinctions in Italian (generally based on morphological and semantic criteria) are often inadequate to represent the syntactic features of words in context. It is worth noting that the uncertainties in categorisation contained in Italian grammars and dictionaries reflect a growing difficulty as they move from fundamental linguistic classes, such as nouns and verbs, to more complex classes, such as adverbs, pronouns, prepositions and conjunctions. This latter class, that groups together elements traditionally used to express connections between sentences, appears inadequate when describing cohesive relations in Italian. This phenomenon actually seems to involve other elements traditionally assigned to different classes, such as adverbs, pronouns and interjections. Recent studies proposed the class of ‘connectives’, grouping all words that, apart from their traditional word class, have the function of connecting phrases and contributing to textual cohesion. From this point of view, conjunctions can be considered as part of phrasal connectives, that can in turn be included in the wider category of textual connectives. The aim of this study is to identify elements that can be included in the class of phrasal connectives, using quantitative methods. According to Shannon and Weaver’s (1949) observation that words are linked by dependent probabilities, corroborated by Halliday’s (1991) argument that the grammatical “system” (in Firth’s sense of the term) is essentially probabilistic, quantitative data are introduced in order to provide evidence of relative frequencies. Section 2 presents a description of word-class categorisation from the point of view of grammars and dictionaries arguing that the traditional category of conjunctions is inadequate for capturing the notion of phrasal connective. Section 3 examines the notion of ‘connective’ and suggests a truth-function interpretation of connective behaviour. Section 4 describes the quantitative methods proposed for analysing the distributional properties of lexical units, and section 5 comments on the results obtained by applying such methods drawing some provisional conclusions. (shrink)

The processing of sequences of (English) sentences is analyzedcompositionally through transitions that merge sentences, rather thandecomposing them. Transitions that are in a precise senseinertial are related to disjunctive and non-deterministic approaches toambiguity. Modal interpretations are investigated, inducing variousequivalences on sequences.

This book constitutes the thoroughly refereed post-proceedings of the Third International Conference on Logical Aspects of Computational Linguistics, LACL'98, held in Grenoble, France, in December 1998. The 15 revised full papers presented together with one invited paper were carefully reviewed and selected during two rounds of refereeing from 33 submissions and 19 conference presentations. Among the topics covered are various types of grammars, categorical inference, automated reasoning, constraint handling, logical forms, dialogue semantics, unification, and proofs.

This book constitutes the refereed proceedings of the 8th International Conference on Conceptual Structures, ICCS 2000, held in Darmstadt, Germany in August 2000. The 38 revised full papers presented have been carefully reviewed and selected for inclusion in the proceedings. The book is divided in topical sections on concepts and language, conceptual onthology, conceptual graphs, formal semantics of conceptual structures, contextual logic and formal concept analysis, conceptual structures in practice, and computational aspects of conceptual structures.

The first of its kind to thoroughly cover language technology at all levels and with all modern technologies this book takes an empirical approach to the ...

There is currently much interest in bringing together the tradition of categorial grammar, and especially the Lambek calculus, with the recent paradigm of linear logic to which it has strong ties. One active research area is designing non-commutative versions of linear logic (Abrusci, 1995; Retoré, 1993) which can be sensitive to word order while retaining the hypothetical reasoning capabilities of standard (commutative) linear logic (Dalrymple et al., 1995). Some connections between the Lambek calculus and computations in groups have long been (...) known (van Benthem, 1986) but no serious attempt has been made to base a theory of linguistic processing solely on group structure. This paper presents such a model, and demonstrates the connection between linguistic processing and the classical algebraic notions of non-commutative free group, conjugacy, and group presentations. A grammar in this model, or G-grammar is a collection of lexical expressions which are products of logical forms, phonological forms, and inverses of those. Phrasal descriptions are obtained by forming products of lexical expressions and by cancelling contiguous elements which are inverses of each other. A G-grammar provides a symmetrical specification of the relation between a logical form and a phonological string that is neutral between parsing and generation modes. We show how the G-grammar can be oriented for each of the modes by reformulating the lexical expressions as rewriting rules adapted to parsing or generation, which then have strong decidability properties (inherent reversibility). We give examples showing the value of conjugacy for handling long-distance movement and quantifier scoping both in parsing and generation. The paper argues that by moving from the free monoid over a vocabulary V (standard in formal language theory) to the free group over V, deep affinities between linguistic phenomena and classical algebra come to the surface, and that the consequences of tapping the mathematical connections thus established can be considerable. (shrink)

Some recent studies in computational linguistics have aimed to take advantage of various cues presented by punctuation marks. This short survey is intended to summarise these research efforts and additionally, to outline a current perspective for the usage and functions of punctuation marks. We conclude by presenting an information-based framework for punctuation, influenced by treatments of several related phenomena in computational linguistics.

In this paper back-and-forth structures are applied to the semantics of natural language. Back-and-forth structures consist of an event structure and an interval structure communicating via a relational link; transitions in the one structure correspond to transitions in the other. Such entities enable us to view temporal constructions (such as tense, aspect, and temporal connectives) as methods of moving systematically between information sources. We illustrate this with a treatment of the English present perfect, and progressive aspect, that draws on ideas (...) developed in Moens & Steedman (1988), and discuss the role of rich ontologies in formal semantics. (shrink)

This book is the first to survey current developments in computational phonology, and it does so in a way that is accessible to computational linguists, phonologists and computer scientists alike.

This paper describes a computational model for analyzing arguments in discourse. In particular, the model describes processes necessary for interpreting one uninterrupted argument from a speaker. The resulting output is a representation for the underlying claim and evidence relations between propositions of the argument. For our processing model we present: (i) a characterization of coherent orderings of propositions, used to limit search for interpretation of each new proposition (ii) a working definition of the evidence relation, used to recognize connections between (...) propositions (iii) a theory of the function and use of clue words — special words and phrases indicating the structure of the argument — then used in the analysis to control search for interpretation and verification of evidence relations. (shrink)

Narrative passages told from a character's perspective convey the character's thoughts and perceptions. We present a discourse process that recognizes characters'.