In this study, we use temporally aligned word embeddings and a large diachronic corpus of English to quantify language change in a data-driven, scalable way, which is grounded in language use. We show a unique and reliable relation between measures of language change and age of acquisition (AoA) while controlling for frequency, contextual diversity, concreteness, length, dominant part of speech, orthographic neighborhood density, and diachronic frequency variation. We analyze measures of language change tackling both the change in (...) class='Hi'>lexical representations and the change in the relation between lexical representations and the words with the most similar usage patterns, showing that they capture different aspects of language change. Our results show a unique relation between language change and AoA, which is stronger when considering neighborhood-level measures of language change: Words with more coherent diachronic usage patterns tend to be acquired earlier. The results support theories positing a link between ontogenetic and ethnogenetic processes in language. (shrink)

Based on a corpus-driven analysis of two translated versions ofHongloumeng(one by David Hawkes and the other by Xianyi Yang and Gladys Yang) in parallel corpora, this article investigates the use of lexical bundles in an attempt to trace the stylistic features and differences in the translations produced by the respective translators. TheHongloumengcorpus is developed at the sentence level to facilitate co-occurrence of the source texts and the two corresponding translations. For this purpose, the three-word and four-word (...) class='Hi'>lexical bundles were first extracted and then analyzed with respect to the functional classification proposed by Biber et al. (2004). The results of the study show that Hawkes' translation is embedded with a greater number and variety of lexical bundles than the one by the Yang couple. The study also identified the differences between the two versions which can be traced back to the deployment of different translation strategies of the translators, appearing in turn to be influenced by the language backgrounds of the translators, the translation skopos and settings, and the social, political, and ideological milieu in which the translations were produced. (shrink)

Grammatical theory has long wrestled with the fact that causative constructions exhibit properties of both single words and complex phrases. However, as Paul Kiparsky has observed, the distribution of such properties of causatives is not arbitrary: ‘construal’ phenomena such as honorification, anaphor and pronominal binding, and quantifier ‘floating’ typically behave as they would if causatives were syntactically complex, embedding constructions; whereas case marking, agreement and word order phenomena all point to the analysis of causatives as single lexical items.1 (...) Although an analysis of causatives in terms of complex syntactic structures has frequently been adopted in an attempt to simplify the mapping to semantic structure, we believe that motivating syntactic structure based on perceived semantics is questionable because in general a syntax/semantics homomorphism cannot be maintained without vitiating syntactic theory (Miller 1991). Instead, we sketch a strictly lexical theory of Japanese causatives that deals with the evidence offered for a complex phrasal analysis. Such an analysis makes the phonology, morphology and syntax parallel, while a mismatch occurs with the semantics. The conclusions we will reach are given in (1). (shrink)

The commentaries provide a multitude of perspectives on the theory of lexical access presented in our target article. We respond, on the one hand, to criticisms that concern the embeddings of our model in the larger theoretical frameworks of human performance and of a speaker's multiword sentence and discourse generation. These embeddings, we argue, are either already there or naturally forgeable. On the other hand, we reply to a host of theory-internal issues concerning the abstract properties of (...) our feedforward spreading activation model, which functions without the usual cascading, feedback, and inhibitory connections. These issues also concern the concrete stratification in terms of lexical concepts, syntactic lemmas, and morphophonology. Our response stresses the parsimony of our modeling in the light of its substantial empirical coverage. We elaborate its usefulness for neuroimaging and aphasiology and suggest further cross-linguistic extensions of the model. (shrink)

Regular polysemes are sets of ambiguous words that all share the same relationship between their meanings, such as CHICKEN and LOBSTER both referring to an animal or its meat. To probe how a distributional semantic model, here exemplified by bidirectional encoder representations from transformers (BERT), represents regular polysemy, we analyzed whether its embeddings support answering sense analogy questions similar to “is the mapping between CHICKEN (as an animal) and CHICKEN (as a meat) similar to that which maps between LOBSTER (...) (as an animal) to LOBSTER (as a meat)?” We did so using the LRcos model, which combines a logistic regression classifier of different categories (e.g., animal vs. meat) with a measure of cosine similarity. We found that (a) the model was sensitive to the shared structure within a given regular relationship; (b) the shared structure varies across different regular relationships (e.g., animal/meat vs. location/organization), potentially reflective of a “regularity continuum;” (c) some high-order latent structure is shared across different regular relationships, suggestive of a similar latent structure across different types of relationships; and (d) there is a lack of evidence for the aforementioned effects being explained by meaning overlap. Lastly, we found that both components of the LRcos model made important contributions to accurate responding and that a variation of this method could yield an accuracy boost of 10% in answering sense analogy questions. These findings enrich previous theoretical work on regular polysemy with a computationally explicit theory and methods, and provide evidence for an important organizational principle for the mental lexicon and the broader conceptual knowledge system. (shrink)

With the increasing prevalence of Internet usage, Internet-Delivered Psychological Treatment (IDPT) has become a valuable tool to develop improved treatments of mental disorders. IDPT becomes complicated and labor intensive because of overlapping emotion in mental health. To create a usable learning application for IDPT requires diverse labeled datasets containing an adequate set of linguistic properties to extract word representations and segmentations of emotions. In medical applications, it is challenging to successfully refine such datasets since emotion-aware labeling is time consuming. (...) Other known issues include vocabulary sizes per class, data source, method of creation, and baseline for the human performance level. This paper focuses on the application of personalized mental health interventions using Natural Language Processing (NLP) and attention-based in-depth entropy active learning. The objective of this research is to increase the trainable instances using a semantic clustering mechanism. For this purpose, we propose a method based on synonym expansion by semantic vectors. Semantic vectors based on semantic information derived from the context in which it appears are clustered. The resulting similarity metrics help to select the subset of unlabeled text by using semantic information. The proposed method separates unlabeled text and includes it in the next active learning mechanism cycle. Our method updates model training by using the new training points. The cycle continues until it reaches an optimal solution, and it converts all the unlabeled text into the training set. Our in-depth experimental results show that the synonym expansion semantic vectors help enhance training accuracy while not harming the results. The bidirectional Long Short-Term Memory (LSTM) architecture with an attention mechanism achieved 0.85 Receiver Operating Characteristic (ROC curve) on the blind test set. The learned embedding is then used to visualize the activated word's contribution to each symptom and find the psychiatrist's qualitative agreement. Our method improves the detection rate of depression symptoms from online forum text using the unlabeled forum texts. (shrink)

The current study represents an integrated model of explicit and implicit offensive language taxonomy. First, it focuses on a definitional revision and enrichment of the explicit offensive language taxonomy by reviewing the collection of available corpora and comparing tagging schemas applied there. The study relies mainly on the categories originally proposed by Zampieri et al. (2019) in terms of offensive language categorization schemata. After the explanation of semantic differences between particular concepts used in the tagging systems and the analysis of (...) theoretical frameworks, a finite set of classes is presented, which cover aspects of offensive language representation along with linguistically sound explanations (Lewandowska-Tomaszczyk et al. 2021). In the analytic procedure, offensive from non-offensive discourse is first distinguished, with the question of offence Target and the following categorization levels and sublevels. Based on the relevant data generated from Sketch Engine (https://www.sketchengine.eu/ententen-english-corpus/), we propose the concept of offensive language as a superordinate category in our system with a number of hierarchically arranged 17 subcategories. The categories are taxonomically structured into 4 levels and verified with the use of neural-based (lexical) embeddings. Together with a taxonomy of implicit offensive language and its subcategorization levels which has received little scholarly attention until now, the categorization is exemplified in samples of offensive discourses in selected English social media materials, i.e., publicly available 25 web-based hate speech datasets (consult Appendix 1 for a complete list). The offensive category levels (types of offence, targets, etc.) and aspects (offensive language property clusters) as well as the categories of explicitness and implicitness are discussed in the study and the computationally verified integrated explicit and implicit offensive language taxonomy proposed in the study. (shrink)

Among theories of human language comprehension, cue‐based memory retrieval has proven to be a useful framework for understanding when and how processing difficulty arises in the resolution of long‐distance dependencies. Most previous work in this area has assumed that very general retrieval cues like [+subject] or [+singular] do the work of identifying (and sometimes misidentifying) a retrieval target in order to establish a dependency between words. However, recent work suggests that general, handpicked retrieval cues like these may not be enough (...) to explain illusions of plausibility (Cunnings & Sturt, 2018), which can arise in sentences like The letter next to the porcelain plate shattered. Capturing such retrieval interference effects requires lexically specific features and retrieval cues, but handpicking the features is hard to do in a principled way and greatly increases modeler degrees of freedom. To remedy this, we use well‐established word embedding methods for creating distributed lexical feature representations that encode information relevant for retrieval using distributed retrieval cue vectors. We show that the similarity between the feature and cue vectors (a measure of plausibility) predicts total reading times in Cunnings and Sturt’s eye‐tracking data. The features can easily be plugged into existing parsing models (including cue‐based retrieval and self‐organized parsing), putting very different models on more equal footing and facilitating future quantitative comparisons. (shrink)

The distinction between literal and figurative use is well-known and embedded in ‘folk linguistics’. According to folk linguistics, figurative uses deviate from literal ones. But recent work on lexical modulation and polysemy shows that meaning deviation is ubiquitous, even in cases of literal use. Hence, it has been argued, the literal/figurative distinction has no value for theorising about communication. In this paper, we focus on metaphor and argue that here the literal–figurative distinction has theoretical importance. The distinction between literal (...) and metaphorical needs to be captured by our account of communication because literal uses transmit information in a way that metaphorical ones do not. We argue that there is a way to explain the literal/metaphorical distinction that preserves the core of the folk-linguistic idea and gives the distinction theoretical relevance. We propose that literal uses of a word are made with the intention to conform to an established practice of use, while metaphorical uses do not so conform, but depend on this pre-existing practice. Our account can deal with data that are problematic for other theories. A further advantage is that it extends naturally to other non-literal uses of words, including metonymy. (shrink)

When a word is used metaphorically (for example “walrus” in the sentence “The president is a walrus”), some features of that word's meaning (“very fat,” “slow-moving”) are carried across to the metaphoric interpretation while other features (“has large tusks,” “lives near the north pole”) are not. What happens to these features that relate only to the literal meaning during processing of novel metaphors? In four experiments, the present study examined the role of the feature of physical containment during (...) processing of verbs of physical containment. That feature is used metaphorically to signify difficulty, such as “fenced in” in the sentence “the journalist's opinion was fenced in after the change in regime.” Results of a lexical decision task showed that video clips displaying a ball being trapped by a box facilitated comprehension of verbs of physical containment when the words were presented in isolation. However, when the verbs were embedded in sentences that rendered their interpretation metaphorical in a novel way, no such facilitation was found, as evidenced by two eye-tracking reading studies. We interpret this as suggesting that features that are critical for understanding the encoded meaning of verbs but are not part of the novel metaphoric interpretation are ignored during the construction of metaphorical meaning. Results and limitations of the paradigm are discussed in relation to previous findings in the literature both on metaphor comprehension and on the interaction between language comprehension and the visual world. (shrink)

A large body of psycholinguistic research demonstrates that both language processing and language acquisition are sensitive to the distributions of linguistic constructions in usage. Here we investigate how statistical distributions at different linguistic levels – morphological and lexical, and phrasal – contribute to the ease with which morphosyntax is processed and produced by second language learners. We analyze Chinese ESL learners’ knowledge of four English inflectional morphemes: -ed, -ing, and third-person -s on verbs, and plural -s on nouns. In (...) Elicited Imitation Tasks, participants listened to length- and difficulty-matched sentences each containing one target morpheme and typed the whole sentence as accurately as they could after a short delay. Experiment 1 investigated lexical and morphemic levels, testing the hypotheses that a morpheme is expected to be more easily processed when it is highly available, and highly reliable. Thirty sentences were made for each morpheme, divided into three Availability-Reliability Distribution groups on the basis of corpus analysis in the Corpus of Contemporary American English : 10 target words high in availability, 10 high in reliability, and 10 low in both reliability and availability. Responses were scored on whether the target morpheme was accurately reproduced given the provision of the correct lemma. A generalized linear mixed-effects logit model revealed fixed effects of morpheme type, availability, and reliability on the accuracy of morpheme provision. There were no effects of lemma frequency. Experiment 2 successfully replicated these results and extended the investigation to explore phrasal formulaicity by manipulating the frequency of the four-word strings in which the morpheme was embedded. GLMMs replicated the effects of word-form availability and reliability and additionally revealed independent phrase-superiority effects where morphemes were better reproduced in contexts of higher string-frequency. Taken together, these findings demonstrate that morpheme acquisition reflects the distributional properties of learners’ experience and the mappings therein between lexis, morphology, phraseology, and semantics. These conclusions support an emergentist view of the statistical symbolic learning of morphology where language acquisition involves the satisfaction of competing constraints across multiple grain-sizes of units. (shrink)

This paper offers a review of a current understanding of the content and the form of linguistic roots. It first updates and buttresses the case against semantic content of uncategorised roots and for Late Insertion of roots; then it investigates how native speakers identify roots. More specifically, the idea that roots may be polysemous or may encode the shadow of a denotation, namely the common denominator of the denotations of words derived from it, is refuted on the basis of conceptual (...) and empirical arguments from a number of languages. Subsequently, the existence of a spectrum of content to which roots belong, with roots ranging from contentless to semantically specific and concrete, is also shown to be illusory, and to result from the actual productivity, hence diversity, of the words derived from it. Arguments for Late Insertion of roots are then reviewed and updated, divorcing roots from the forms that realise them. These arguments are systematically combined with the semantic contentlessness of roots in support of Acquaviva’s analysis of them as abstract indices, i.e. as the syntax-internal criteria of lexical identity. This account is taken to its logical conclusion in the final section: if roots are indeed abstract indices, then they cannot be identified either by the semantic content they realise within grammatical structures or by their forms. An account is therefore advanced according to which roots are identified just once by native speakers over their lexicon at a given moment and on the basis of three heuristic principles: one form-based, one based on the feature content and the exponence of the structures in which roots are embedded, and one taking care of root suppletion. (shrink)

The automated identification of national implementations of European directives by text similarity techniques has shown promising preliminary results. Previous works have proposed and utilized unsupervised lexical and semantic similarity techniques based on vector space models, latent semantic analysis and topic models. However, these techniques were evaluated on a small multilingual corpus of directives and NIMs. In this paper, we utilize word and paragraph embedding models learned by shallow neural networks from a multilingual legal corpus of European directives and (...) national legislation to develop unsupervised semantic similarity systems to identify transpositions. We evaluate these models and compare their results with the previous unsupervised methods on a multilingual test corpus of 43 Directives and their corresponding NIMs. We also develop supervised machine learning models to identify transpositions and compare their performance with different feature sets. (shrink)

The automated identification of national implementations of European directives by text similarity techniques has shown promising preliminary results. Previous works have proposed and utilized unsupervised lexical and semantic similarity techniques based on vector space models, latent semantic analysis and topic models. However, these techniques were evaluated on a small multilingual corpus of directives and NIMs. In this paper, we utilize word and paragraph embedding models learned by shallow neural networks from a multilingual legal corpus of European directives and (...) national legislation to develop unsupervised semantic similarity systems to identify transpositions. We evaluate these models and compare their results with the previous unsupervised methods on a multilingual test corpus of 43 Directives and their corresponding NIMs. We also develop supervised machine learning models to identify transpositions and compare their performance with different feature sets. (shrink)

Sentence comprehension involves maintaining and continuously integrating linguistic information and, thus, makes demands on working memory. Past research has demonstrated that semantic WM, but not phonological WM, is critical for integrating word meanings across some distance and resolving semantic interference in sentence comprehension. Here, we examined the relation between phonological and semantic WM and the comprehension of center-embedded relative clause sentences, often argued to make heavy demands on WM. Additionally, we examined the relation between phonological and semantic WM and (...) the comprehension of transitive and dative active and passive sentences, which may also draw on WM resources depending on the number of propositions that must be maintained and the difficulty of processing passive clauses. In a large sample of individuals with aphasia, we assessed whether comprehension performance on more complex vs. simpler active-passive or embedded relative clause sentences would be predicted by semantic but not phonological WM when controlling for single word comprehension. For performance on the active-passive comprehension task, we found that semantic WM, but not phonological WM, predicted comprehension of dative sentences when controlling for comprehension of transitive sentences. We also found that phonological WM, but not semantic WM, predicted mean comprehension for reversible active-passive sentences when controlling for trials with lexical distractors. On the relative clause comprehension task, consistent with prior results, we found that semantic WM, but not phonological WM, predicted comprehension of object relative clause sentences and relative clause sentences with a passive construction. However, both phonological WM and semantic WM predicted mean comprehension across all relative clause types for reversible trials when controlling for trials with lexical distractors. While we found evidence of semantic WM’s role in comprehension, we also observed unpredicted relations between phonological WM and comprehension in some conditions. Post-hoc analyses provided preliminary evidence that phonological WM maintains a backup phonological representation of the sentence that may be accessed when sentence comprehension processing is less efficient. Future work should investigate possible roles that phonological WM may play across sentence types. (shrink)

Job seekers find themselves increasingly duped and misled by fraudulent job advertisements, posing a threat to their privacy, security and well-being. There is a clear need for solutions that can protect innocent job seekers. Existing approaches to detecting fraudulent jobs do not scale well, function like a black-box, and lack interpretability, which is essential to guide applicants’ decision-making. Moreover, commonly used lexical features may be insufficient as the representation does not capture contextual semantics of the underlying document. Hence, this (...) paper explores to what extent different categorizations of fraudulent jobs can be classified. In addition, this paper seeks to find what type of features are most relevant in classifying the type of fraudulent job. In this paper, we develop and validate a machine learning system for identifying identity theft, corporate identity theft and multi-level marketing amongst fraudulent job advertisements. We utilized four classes of features: empirical rule set-based features, bag-of-word models, most recent state-of-the-art word embeddings and transformer models for various machine learning classifiers. The machine learning models were validated by evaluating them on a publicly available job description dataset. Our results indicate that the word embeddings and transformer-based features consistently outperformed the handcrafted rule-set based features class. Ultimately, a Gradient Boosting classifier with a combination of empirical rule-set based features, parts-of-speech tags and bag-of-words vectors achieved the best performance with an F1-score of 0.88. (shrink)