We explore a network architecture introduced by Elman (1988) for predicting successive elements of a sequence. The network uses the pattern of activation over a set of hidden units from time-step 25-1, together with element t, to predict element t + 1. When the network is trained with strings from a particular finite-state grammar, it can learn to be a perfect finite-state recognizer for the grammar. When the network has a minimal number of hidden units, patterns (...) on the hidden units come to correspond to the nodes of the grammar, although this correspondence is not necessary for the network to act as a perfect finite-state recognizer. We explore the conditions under which the network can carry information about distant sequential contingencies across intervening elements. Such information is maintained with relative ease if it is.. (shrink)

Although many studies have provided evidence that abstract knowledge can be acquired in artificial grammar learning, it remains unclear how abstract knowledge can be attained in sequence learning. To address this issue, we proposed a dual simple recurrent network model that includes a surface SRN encoding and predicting the surface properties of stimuli and an abstract SRN encoding and predicting the abstract properties of stimuli. The results of Simulations 1 and 2 showed that the DSRN model can (...) account for learning effects in the serial reaction time task under different conditions, and the manipulation of the contribution weight of each SRN accounted for the contribution of conscious and unconscious processes in inclusion and exclusion tests in previous studies. The results of human performance in Simulation 3 provided further evidence that people can implicitly learn both chunking and abstract knowledge in sequence learning, and the results of Simulation 3 confirmed that the DSRN model can account for how people implicitly acquire the two types of knowledge in sequence learning. These findings extend the learning ability of the SRN model and help understand how different types of knowledge can be acquired implicitly in sequence learning. (shrink)

Timely detection and efficient recognition of fault are challenging for the bogie of high-speed train, owing to the fact that different types of fault signals have similar characteristics in the same frequency range. Notice that convolutional neural networks are powerful in extracting high-level local features and that recurrent neural networks are capable of learning long-term context dependencies in vibration signals. In this paper, by combining CNN and RNN, a so-called convolutional recurrent neural network is proposed to diagnose (...) various faults of the HST bogie, where the capabilities of CNN and RNN are inherited simultaneously. Within the novel architecture, the proposed CRNN first filters out the features from the original data through convolutional layers. Then, four recurrent layers with simple recurrent cell are used to model the context information in the extracted features. By comparing the performance of the presented CRNN with CNN, RNN, and ensemble learning, experimental results show that CRNN achieves not only the best performance with accuracy of 97.8% but also the least time spent in training model. (shrink)

Prediction‐based processes appear to play an important role in language. Few studies, however, have sought to test the relationship within individuals between prediction learning and natural language processing. This paper builds upon existing statistical learning work using a novel paradigm for studying the on‐line learning of predictive dependencies. Within this paradigm, a new “prediction task” is introduced that provides a sensitive index of individual differences for developing probabilistic sequential expectations. Across three interrelated experiments, the prediction task and results thereof are (...) used to bridge knowledge of the empirical relation between statistical learning and language within the context of nonadjacency processing. We first chart the trajectory for learning nonadjacencies, documenting individual differences in prediction learning. Subsequent simple recurrent network simulations then closely capture human performance patterns in the new paradigm. Finally, individual differences in prediction performances are shown to strongly correlate with participants’ sentence processing of complex, long‐distance dependencies in natural language. (shrink)

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine, a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can (...) encode contextual information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models. We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain. (shrink)

Current cognitive science models of perception and action assume that the objects that we move toward and perceive are represented as determinate in our experience of them. A proper phenomenology of perception and action, however, shows that we experience objects indeterminately when we are perceiving them or moving toward them. This indeterminacy, as it relates to simple movement and perception, is captured in the proposed phenomenologically based recurrent network models of brain function. These models provide a possible (...) foundation from which predicative structures may arise as an emergent phenomenon without the positing of a representing subject. These models go some way in addressing the dual constraints of phenomenological accuracy and neurophysiological plausibility that ought to guide all projects devoted to discovering the physical basis of human experience. (shrink)

Some argue the common practice of inferring multiple processes or systems from a dissociation is flawed. One proposed solution is state-trace analysis, which involves plotting, across two or more conditions of interest, performance measured by either two dependent variables, or two conditions of the same dependent measure. The resulting analysis is considered to provide evidence that either a single process underlies performance or there is evidence for more than one process. This article reports simulations using the simple recurrent (...) network in which changes to the learning rate produced state-trace plots with multiple functions. We also report simulations using a single-layer error-correcting network that generate plots with a single function. We argue that the presence of different functions on a state-trace plot does not necessarily support a dual-system account, at least as typically defined ; it can also indicate variation in a single parameter within theories generally considered to be single-system accounts. (shrink)

We examine two connectionist networks—a fractal learning neural network (FLNN) and a Simple Recurrent Network (SRN)—that are trained to process center-embedded symbol sequences. Previous work provides evidence that connectionist networks trained on infinite-state languages tend to form fractal encodings. Most such work focuses on simple counting recursion cases (e.g., anbn), which are not comparable to the complex recursive patterns seen in natural language syntax. Here, we consider exponential state growth cases (including mirror recursion), describe a (...) new training scheme that seems to facilitate learning, and note that the connectionist learning of these cases has a continuous metamorphosis property that looks very different from what is achievable with symbolic encodings. We identify a property—ragged progressive generalization—which helps make this difference clearer. We suggest two conclusions. First, the fractal analysis of these more complex learning cases reveals the possibility of comparing connectionist networks and symbolic models of grammatical structure in a principled way—this helps remove the black box character of connectionist networks and indicates how the theory they support is different from symbolic approaches. Second, the findings indicate the value of future, linked mathematical and empirical work on these models—something that is more possible now than it was 10 years ago. (shrink)

The amount of information that is produced on a daily basis in the financial markets is vast and complex; consequently, the development of systems that simplify decision-making is an essential endeavor. In this article, several intelligent systems are proposed and tested to predict the closing price of the IBEX 35 index using more than ten years of historical data and five distinct architectures for neural networks. A multi-layer perceptron was the first step, followed by a simple recurrent neural (...) network, a gated recurrent unit network and two long-short-term memory (LSTM) networks. The results of the analyses performed on these models have demonstrated a powerful capacity for prediction. Additionally, the findings of this research point to the fact that the application of intelligent systems can simplify the decision-making process in financial markets, which is a substantial advantage. Furthermore, by comparing the predicted outcome errors between the models, the LSTM presents the lowest error with a higher computational time in the training phase. The LSTM was able to accurately forecast the closing price of the day as well as the price for the following one and two days in advance. In conclusion, the empirical results demonstrated that these models could accurately predict financial data for trading purposes and that the application of intelligent systems, such as the LSTM network, represents a promising advancement in financial technology. (shrink)

Human cognition is said to be systematic: cognitive ability generalizes to structurally related behaviours. The connectionist approach to cognitive theorizing has been strongly criticized for its failure to explain systematicity. Demonstrations of generalization notwithstanding, I show that two widely used networks (feedforward and recurrent) do not support systematicity under the condition of local input/output representations. For a connectionist explanation of systematicity, these results leave two choices, either: (1) develop models capable of systematicity under local input/output representations; or (2) justify (...) the choice of similarity-based (nonlocal) component representations sufficient for systematicity. (shrink)

Comparing the relative sensitivity of direct and indirect measures of learning is proposed as the best way to provide evidence for unconscious learning when both conceptual and operative definitions of awareness are lacking. This approach was first proposed by Reingold & Merikle (1988) in the context of subliminal perception. In this paper, we apply it to a choice reaction time task in which the material is generated based on a probabilistic finite-state grammar (Cleeremans, 1993). We show (1) that participants progressively (...) learn about the statistical structure of the stimulus material over training with the choice reaction time task, and (2) that they can use some of this knowledge to predict the location of the next stimulus in a subsequent “generation” task. However, detailed partial correlational analyses of the correspondence between performance during the reaction time task and the statistical structure of the training material showed that large effects remained even when controlling for explicit knowledge as assessed by the generation task. Hence we conclude (1) that at least some of the knowledge expressed through reaction time performance can not be characterized as conscious, and (2) that even when associations are found at a global level of analysis, dissociations can still be obtained when more detailed analyses are conducted. Finally, we also show that participants are limited in the depth of the contingencies they can learn about, and that these limitations are shared by the Simple Recurrent Network model of Cleeremans & McClelland (1991). (shrink)

A simple recurrent artificial neural network is used to classify situations as permissible or impermissible. The trained ANN can be understood as having set up a similarity space of cases at the level of its internal or hidden units. An analysis of the network’s internal representations is undertaken using a new visualization technique for state space approaches to understanding similarity. Insights from the literature on moral philosophy pertaining to contributory standards will be used to interpret the (...) state space set up by the ANN as being structured by implicit reasons. The ANN, on its own, is not capable of explicitly representing or offering reasons to itself or others. That said, the low level similarity space set up by the network could be made available to higher order processes that exploit it for case-based reasoning. It is argued that for normative purposes, similarity could be seen as a contributor to procedural coherence in case-based reasoning and local forms of substantive coherence, but not to global forms of coherence given the computational complexity of managing those more ambitious forms of coherence. (shrink)

Over the past decade, sequence learning has gradually become a central paradigm through which to study implicit learning. In this chapter, we start by briefly summarizing the results obtained with different variants of the sequence learning paradigm. We distinguish three subparadigms in terms of whether the stimulus material is generated either by following a fixed and repeating sequence (e.g., Nissen & Bullemer, 1987), by relying on a complex set of rules from which one can produce several alternative deterministic sequences (e.g., (...) Lewicki, Hill & Bizot, 1988; Stadler, 1989), or by following the output of a probabilistic set of rules such as instantiated by noisy finite-state grammars (Cleeremans & McClelland, 1991; Jiménez, Mendéz & Cleeremans, 1996). Next, we focus on the processes involved in sequence representation and acquisition. We suggest that the sensitivity to the sequential structure observed in the probabilistic subparadigm can only be a result of the acquisition of a representation of the statistical constraints of the material, and that this sensitivity emerges through the operation of mechanisms that are well instantiated by connectionist models such as the Simple Recurrent Network (Elman, 1990; Cleeremans, 1993b). We present new simulation work meant to explore to what extent the model can also account for specific data obtained in a paradigmatic instance of deterministic, rule-based sequence learning task: Lewicki et al. (1988)'s situation. Finally, we report on the results of an experiment that compares learning on otherwise similar deterministic and probabilistic structures, and we show that learning of both types of structures is equivalent only under conditions that maximally hinder explicit acquisition. Taken together, these simulation and experimental data lend support to the claim that implicit learning in all three sequence learning subparadigms can amount to a form of statistical sequence learning. They also suggest that distinguishing among several theories of sequence representation and acquisition may require us to analize the data in great detail. Hopefully, however, some truth can be found in such details.. (shrink)

How people learn chunks or associations between adjacent items in sequences was modelled. Two previously successful models of how people learn artificial grammars were contrasted: the CCN, a network version of the competitive chunker of Servan‐Schreiber and Anderson [J. Exp. Psychol.: Learn. Mem. Cogn. 16 (1990) 592], which produces local and compositionally‐structured chunk representations acquired incrementally; and the simple recurrent network (SRN) of Elman [Cogn. Sci. 14 (1990) 179], which acquires distributed representations through error correction. The (...) models' susceptibility to two types of interference was determined: prediction conflicts, in which a given letter can predict two other letters that appear next with an unequal frequency; and retroactive interference, in which the prediction made by a letter changes in the second half of training. The predictions of the models were determined by exploring parameter space and seeing howdensely different regions of the space of possible experimental outcomes were populated by model outcomes. For both types of interference, human data fell squarely in regions characteristic of CCN performance but not characteristic of SRN performance. (shrink)

A significant part of everyday learning occurs incidentally — a process typically described as implicit learning. A central issue in this domain and others, such as language acquisition, is the extent to which performance depends on the acquisition and deployment of abstract rules. Shanks and colleagues [22], [11] have suggested (1) that discrimination between grammatical and ungrammatical instances of a biconditional grammar requires the acquisition and use of abstract rules, and (2) that training conditions — in particular whether instructions orient (...) participants to identify the relevant rules or not — strongly influence the extent to which such rules will be learned. In this paper, we show (1) that a Simple Recurrent Network can in fact, under some conditions, learn a biconditional grammar, (2) that training conditions indeed influence learning in simple auto-associators networks and (3) that such networks can likewise learn about biconditional grammars, albeit to a lesser extent than human participants. These findings suggest that mastering biconditional grammars does not require the acquisition of abstract rules to the extent implied by Shanks and colleagues, and that performance on such material may in fact be based, at least in part, on simple associative learning mechanisms. (shrink)

The demonstration of a sequential congruency effect in sequence learning has been offered as evidence for control processes that act to inhibit automatic response tendencies via unconscious conflict monitoring. Here we propose an alternative interpretation of this effect based on the associative learning of chains of sequenced contingencies. This account is supported by simulations with a Simple Recurrent Network, an associative model of sequence learning. We argue that the control- and associative-based accounts differ in their predictions concerning (...) the magnitude of the sequential congruency effect across training. These predictions are tested by reanalysing data from a study by Shanks, Wilkinson, and Channon . The results support the associative learning account which explains the sequential congruency effect without appealing to control processes. (shrink)

We identify a number of principles with respect to prediction that, we argue, underpin adult language comprehension: (a) comprehension consists in realizing a mapping between the unfolding sentence and the event representation corresponding to the real‐world event being described; (b) the realization of this mapping manifests as the ability to predict both how the language will unfold, and how the real‐world event would unfold if it were being experienced directly; (c) concurrent linguistic and nonlinguistic inputs, and the prior internal states (...) of the system, each drive the predictive process; (d) the representation of prior internal states across a representational substrate common to the linguistic and nonlinguistic domains enables the predictive process to operate over variable time frames and variable levels of representational abstraction. We review empirical data exemplifying the operation of these principles and discuss the relationship between prediction, event structure, thematic role assignment, and incrementality. (shrink)

As linguistic competence so clearly illustrates, processing sequences of events is a fundamental aspect of human cognition. For this reason perhaps, sequence learning behavior currently attracts considerable attention in both cognitive psychology and computational theory. In typical sequence learning situations, participants are asked to react to each element of sequentially structured visual sequences of events. An important issue in this context is to determine whether essentially associative processes are sufficient to understand human performance, or whether more powerful learning mechanisms are (...) necessary. To address this issue, we explore how well human participants and connectionist models are capable of learning sequential material that involves complex, disjoint, longdistance contingencies. We show that the popular Simple Recurrent Network model (Elman, 1990), which has otherwise been shown to account for a variety of empirical findings (Cleeremans, 1993), fails to account for human performance in several experimental situations meant to test the model’s specific predictions. In previous research (Cleeremans, 1993) briefly described in this paper, the structure of center-embedded sequential structures was manipulated to be strictly identical or probabilistically different as a function of the elements surrounding the embedding. While the SRN could only learn in the second case, human subjects were found to be insensitive to the manipulation. In the new experiment described in this paper, we tested the idea that performance benefits from “starting small effects” (Elman, 1993) by contrasting two conditions in which the training regimen was either incremental or not. Again, while the SRN is only capable of learning in the first case, human subjects were able to learn in both. We suggest an alternative model based on Maskara & Noetzel’s (1991) Auto-Associative Recurrent Network as a way to overcome the SRN model’s failure to account for the empirical findings.. (shrink)

Comparing the relative sensitivity of direct and indirect measures of learning is proposed as the best way to provide evidence for unconscious learning when both conceptual and operative definitions of awareness are lacking. This approach was first proposed by Reingold & Merikle (1988) in the context of subliminal perception. In this paper, we apply it to a choice reaction time task in which the material is generated based on a probabilistic finite-state grammar (Cleeremans, 1993). We show (1) that participants progressively (...) learn about the statistical structure of the stimulus material over training with the choice reaction time task, and (2) that they can use some of this knowledge to predict the location of the next stimulus in a subsequent “generation” task. However, detailed partial correlational analyses of the correspondence between performance during the reaction time task and the statistical structure of the training material showed that large effects remained even when controlling for explicit knowledge as assessed by the generation task. Hence we conclude (1) that at least some of the knowledge expressed through reaction time performance can not be characterized as conscious, and (2) that even when associations are found at a global level of analysis, dissociations can still be obtained when more detailed analyses are conducted. Finally, we also show that participants are limited in the depth of the contingencies they can learn about, and that these limitations are shared by the Simple Recurrent Network model of Cleeremans & McClelland (1991). (shrink)

Although for many years a sharp distinction has been made in language research between rules and words—with primary interest on rules—this distinction is now blurred in many theories. If anything, the focus of attention has shifted in recent years in favor of words. Results from many different areas of language research suggest that the lexicon is representationally rich, that it is the source of much productive behavior, and that lexically specific information plays a critical and early role in the interpretation (...) of grammatical structure. But how much information can or should be placed in the lexicon? This is the question I address here. I review a set of studies whose results indicate that event knowledge plays a significant role in early stages of sentence processing and structural analysis. This poses a conundrum for traditional views of the lexicon. Either the lexicon must be expanded to include factors that do not plausibly seem to belong there; or else virtually all information about word meaning is removed, leaving the lexicon impoverished. I suggest a third alternative, which provides a way to account for lexical knowledge without a mental lexicon. (shrink)

Sibley et al. (2008) report a recurrent neural network model designed to learn wordform representations suitable for written and spoken word identification. The authors claim that their sequence encoder network overcomes a key limitation associated with models that code letters by position (e.g., CAT might be coded as C‐in‐position‐1, A‐in‐position‐2, T‐in‐position‐3). The problem with coding letters by position (slot‐coding) is that it is difficult to generalize knowledge across positions; for example, the overlap between CAT and TOMCAT is (...) lost. Although we agree this is a critical problem with many slot‐coding schemes, we question whether the sequence encoder model addresses this limitation, and we highlight another deficiency of the model. We conclude that alternative theories are more promising. (shrink)

At root, the systematicity debate over classical versus connectionist explanations for cognitive architecture turns on quantifying the degree to which human cognition is systematic. We introduce into the debate recent psychological data that provides strong support for the purely structure-based generalizations claimed by Fodor and Pylyshyn (1988). We then show, via simulation, that two widely used connectionist models (feedforward and simple recurrent networks) do not capture the same degree of generalization as human subjects. However, we show that this (...) limitation is overcome by tensor networks that support relational processing. (shrink)

A number of recent studies have examined the effects of phonological variation on the perception of speech. These studies show that both the lexical representations of words and the mechanisms of lexical access are organized so that natural, systematic variation is tolerated by the perceptual system, while a general intolerance of random deviation is maintained. Lexical abstraction distinguishes between phonetic features that form the invariant core of a word and those that are susceptible to variation. Phonological inference relies on the (...) context of surface changes to retrieve the underlying phonological form. In this article we present a model of these processes in speech perception, based on connectionist learning techniques. A simple recurrent network was trained on the mapping from the variant surface form of speech to the underlying form. Once trained, the network exhibited features of both abstraction and inference in its processing of normal speech, and predicted that similar behavior will be found in the perception of nonsense words. This prediction was confirmed in subsequent research (Gaskell & Marslen-Wilson, 1994). (shrink)

A significant part of everyday learning occurs incidentally — a process typically described as implicit learning. A central issue in this domain and others, such as language acquisition, is the extent to which performance depends on the acquisition and deployment of abstract rules. Shanks and colleagues [22], [11] have suggested (1) that discrimination between grammatical and ungrammatical instances of a biconditional grammar requires the acquisition and use of abstract rules, and (2) that training conditions — in particular whether instructions orient (...) participants to identify the relevant rules or not — strongly influence the extent to which such rules will be learned. In this paper, we show (1) that a Simple Recurrent Network can in fact, under some conditions, learn a biconditional grammar, (2) that training conditions indeed influence learning in simple auto-associators networks and (3) that such networks can likewise learn about biconditional grammars, albeit to a lesser extent than human participants. These findings suggest that mastering biconditional grammars does not require the acquisition of abstract rules to the extent implied by Shanks and colleagues, and that performance on such material may in fact be based, at least in part, on simple associative learning mechanisms. (shrink)

A significant part of everyday learning occurs incidentally — a process typically described as implicit learning. A central issue in this and germane domains such as language acquisition is the extent to which performance depends on the acquisition and deployment of abstract rules. In an attempt to address this question, we show that the apparent use of such rules in a simple categorisation task of artificial grammar strings, as reported by Shanks, Johnstone, and Staggs (1997), can be simulated by (...) means of a simple recurrent network, and may thus turn out not be incompatible with the acquisition of statistical regularities rooted in the processing of exemplars of the presented material. (shrink)

The forms of words as they appear in text and speech are central to theories and models of lexical processing. Nonetheless, current methods for simulating their learning and representation fail to approach the scale and heterogeneity of real wordform lexicons. A connectionist architecture termed thesequence encoderis used to learn nearly 75,000 wordform representations through exposure to strings of stress‐marked phonemes or letters. First, the mechanisms and efficacy of the sequence encoder are demonstrated and shown to overcome problems with traditional slot‐based (...) codes. Then, two large‐scale simulations are reported that learned to represent lexicons of either phonological or orthographic wordforms. In doing so, the models learned the statistics of their lexicons as shown by better processing of well‐formed pseudowords as opposed to ill‐formed (scrambled) pseudowords, and by accounting for variance in well‐formedness ratings. It is discussed how the sequence encoder may be integrated into broader models of lexical processing. (shrink)

The forms of words as they appear in text and speech are central to theories and models of lexical processing. Nonetheless, current methods for simulating their learning and representation fail to approach the scale and heterogeneity of real wordform lexicons. A connectionist architecture termed thesequence encoderis used to learn nearly 75,000 wordform representations through exposure to strings of stress‐marked phonemes or letters. First, the mechanisms and efficacy of the sequence encoder are demonstrated and shown to overcome problems with traditional slot‐based (...) codes. Then, two large‐scale simulations are reported that learned to represent lexicons of either phonological or orthographic wordforms. In doing so, the models learned the statistics of their lexicons as shown by better processing of well‐formed pseudowords as opposed to ill‐formed (scrambled) pseudowords, and by accounting for variance in well‐formedness ratings. It is discussed how the sequence encoder may be integrated into broader models of lexical processing. (shrink)

Marcus et al.’s experiment (1999) concerning infant ability to distinguish between differing syntactic structures has prompted connectionists to strive to show that certain types of neural networks can mimic the infants’ results. In this paper we take a closer look at two such attempts: Shultz and Bale [Shultz, T.R. and Bale, A.C. (2001), Infancy 2, pp. 501–536] Altmann and Dienes [Altmann, G.T.M. and Dienes, Z. (1999) Science 248, p. 875a]. We were not only interested in how well these two models (...) matched the infants’ results, but also whether they were genuinely learning the grammars involved in this process. After performing an extensive set of experiments, we found that, at first blush, Shultz and Bale’s model (2001) replicated the infant’s known data, but the model largely failed to learn the grammars. We also found serious problems with Altmann and Dienes’ model (1999), which fell short of matching any of the infant’s results and of learning the syntactic structure of the input patterns. (shrink)

The approach used by physics is based on the identification and study of ideal objects, which is also the basis of biophysics, in combination with von Neumann heuristic modeling and functional fractionation according to R.Rosen is discussed as a tool for studying the properties of consciousness. The object of the study is a kind of line of analog systems: the human brain, the vertebrate brain, the invertebrate brain and artificial neural networks capable of reflection, which is a key property characteristic (...) of consciousness. Reflection in the broad sense of the word, understood as an internal representation of the external world, is characteristic of a wide range of animals, and some of them (bumblebees, fish) even demonstrate reflection in the narrow sense of the word, understood as an inner self-representation. This complex behavior is realized by miniature brains of ~1 million neurons. The use of simple recurrent neural networks (RNNs) to obtain answers to general questions is illustrated. For example, it has been shown a small RNS is able to pass delayed matching to sample (DMTS) test, forming an individual dynamic representation of the received stimulus, allowing decoding by a special external neural detector.. It has been demonstrated in the reflexive game “even-odd”, the RNS has a huge advantage over a multi-layered neural network, with the same and a larger number of neurons – reflection defeats regression. It was found that the asymmetry of outcomes in the odd-even game, which was explained by various causes, including psychological ones – “it’s easier to catch up than to run away”, is reproduced in the game of two RNNs. Obviously, there are no psychological causes here and the advantage of the player playing for “even” is explained by the more complex strategy of the “odd” player – he needs to predict the opponent’s move and choose the opposite one. (shrink)

This paper proposes several recurrent neural network-based models for recognizing requisite and effectuation parts in Legal Texts. Firstly, we propose a modification of BiLSTM-CRF model that allows the use of external features to improve the performance of deep learning models in case large annotated corpora are not available. However, this model can only recognize RE parts which are not overlapped. Secondly, we propose two approaches for recognizing overlapping RE parts including the cascading approach which uses the sequence of (...) BiLSTM-CRF models and the unified model approach with the multilayer BiLSTM-CRF model and the multilayer BiLSTM-MLP-CRF model. Experimental results on two Japan law RRE datasets demonstrated advantages of our proposed models. For the Japanese National Pension Law dataset, our approaches obtained an \ score of 93.27% and exhibited a significant improvement compared to previous approaches. For the Japan Civil Code RRE dataset which is written in English, our approaches produced an \ score of 78.24% in recognizing RE parts that exhibited a significant improvement over strong baselines. In addition, using external features and in-domain pre-trained word embeddings also improved the performance of RRE systems. (shrink)

‘Particularism’ and ‘generalism’ refer to families of positions in the philosophy of moral reasoning, with the former playing down the importance of principles, rules or standards, and the latter stressing their importance. Part of the debate has taken an empirical turn, and this turn has implications for AI research and the philosophy of cognitive modeling. In this paper, Jonathan Dancy’s approach to particularism (arguably one of the best known and most radical approaches) is questioned both on logical and empirical grounds. (...) Doubts are raised over whether Dancy’s brand of particularism can adequately explain the graded nature of similarity assessments in analogical arguments. Also, simple recurrent neural network models of moral case classification are presented and discussed. This is done to raise concerns about Dancy’s suggestion that neural networks can help us to understand how we could classify situations in a way that is compatible with his particularism. Throughout, the idea of a surveyable standard—one with restricted length and complexity—plays a key role. Analogical arguments are taken to involve multidimensional similarity assessments, and surveyable contributory standards are taken to be attempts to articulate the dimensions of similarity that may exist between cases. This work will be of relevance both to those who have interests in computationally modeling human moral cognition and to those who are interested in how such models may or may not improve our philosophical understanding of such cognition. (shrink)

The approach used by physics is based on the identification and study of ideal objects, which is also the basis of biophysics, in combination with von Neumann heuristic modeling and functional fractionation according to R.Rosen is discussed as a tool for studying the properties of consciousness. The object of the study is a kind of line of analog systems: the human brain, the vertebrate brain, the invertebrate brain and artificial neural networks capable of reflection, which is a key property characteristic (...) of consciousness. Reflection in the broad sense of the word, understood as an internal representation of the external world, is characteristic of a wide range of animals, and some of them (bumblebees, fish) even demonstrate reflection in the narrow sense of the word, understood as an inner self-representation. This complex behavior is realized by miniature brains of ~1 million neurons. The use of simple recurrent neural networks (RNNs) to obtain answers to general questions is illustrated. For example, it has been shown a small RNS is able to pass delayed matching to sample (DMTS) test, forming an individual dynamic representation of the received stimulus, allowing decoding by a special external neural detector.. It has been demonstrated in the reflexive game “even-odd”, the RNS has a huge advantage over a multi-layered neural network, with the same and a larger number of neurons – reflection defeats regression. It was found that the asymmetry of outcomes in the odd-even game, which was explained by various causes, including psychological ones – “it’s easier to catch up than to run away”, is reproduced in the game of two RNNs. Obviously, there are no psychological causes here and the advantage of the player playing for “even” is explained by the more complex strategy of the “odd” player – he needs to predict the opponent’s move and choose the opposite one. (shrink)

There is increasing interest in how the pupil dynamics of the eye reflect underlying cognitive processes and brain states. Problematic, however, is that pupil changes can be due to non-cognitive factors, for example luminance changes in the environment, accommodation and movement. In this paper we consider how by modeling the response of the pupil in real-world environments we can capture the non-cognitive related changes and remove these to extract a residual signal which is a better index of cognition and performance. (...) Specifically, we utilize sequence measures such as fixation position, duration, saccades, and blink-related information as inputs to a deep recurrent neural network (RNN) model for predicting subsequent pupil diameter. We build and evaluate the model for a task where subjects are watching educational videos and subsequently asked questions based on the content. Compared to commonly-used models for this task, the RNN had the lowest errors rates in predicting subsequent pupil dilation given sequence data. Most importantly was how the model output related to subjects' cognitive performance as assessed by a post-viewing test. Consistent with our hypothesis that the model captures non-cognitive pupil dynamics, we found (1) the model's root-mean square error was less for lower performing subjects than for those having better performance on the post-viewing test, (2) the residuals of the RNN (LSTM) model had the highest correlation with subject post-viewing test scores and (3) the residuals had the highest discriminability (assessed via area under the ROC curve, AUC) for classifying high and low test performers, compared to the true pupil size or the RNN model predictions. This suggests that deep learning sequence models may be good for separating components of pupil responses that are linked to luminance and accommodation from those that are linked to cognition and arousal. (shrink)

It has been shown that Recurrent Artificial Neural Networks automatically acquire some grammatical knowledge in the course of performing linguistic prediction tasks. The extent to which such networks can actually learn grammar is still an object of investigation. However, being mostly data-driven, they provide a natural testbed for usage-based theories of language acquisition. This mini-review gives an overview of the state of the field, focusing on the influence of the theoretical framework in the interpretation of results.

This paper examines flexibility in ad hoc networks and suggests that, even with cross-layer design as a mechanism to improve adaptation, a fundamental limitation exists in the ability of a single optimization function, defined a priori, to adapt the network to meet all quality-of-service requirements. Thus, code implementing multiple algorithms will have to be positioned within the network. Active networking and programmable networking enable unprecedented autonomy and flexibility for ad hoc communication networks. However, in order to best leverage (...) the results of active and programmable networking, metrics that indicate the nature and location of required flexibility need to be developed. The primary contribution of this paper is to propose a metric that couples network topological rate of change with the ability of a generic service to move itself to an optimal location in concert with the changing network. This metric points to a fundamental tradeoff among adaptation (changing service location), performance (sophistication or estimated minimum code size of the service), and the network’s ability to tune itself to a changing ad hoc network topology. (shrink)

The development of analogical reasoning has traditionally been understood in terms of theories of adult competence. This approach emphasizes structured representations and structure mapping. In contrast, we argue that by taking a developmental perspective, analogical reasoning can be viewed as the product of a substantially different cognitive ability – relational priming. To illustrate this, we present a computational (here connectionist) account where analogy arises gradually as a by-product of pattern completion in a recurrent network. Initial exposure to a (...) situation primes a relation that can then be applied to a novel situation to make an analogy. Relations are represented as transformations between states. The network exhibits behaviors consistent with a broad range of key phenomena from the developmental literature, lending support to the appropriateness of this approach (using low-level cognitive mechanisms) for investigating a domain that has normally been the preserve of high-level models. Furthermore, we present an additional simulation that integrates the relational priming mechanism with deliberative controlled use of inhibition to demonstrate how the framework can be extended to complex analogical reasoning, such as the data from explicit mapping studies in the literature on adults. This account highlights how taking a developmental perspective constrains the theory construction and cognitive modeling processes in a way that differs substantially from that based purely on adult studies, and illustrates how a putative complex cognitive skill can emerge out of a simple mechanism. (shrink)

Computer simulations show that an unstructured neural-network model [Shultz, T. R., & Bale, A. C. (2001). Infancy, 2, 501–536] covers the essential features␣of infant learning of simple grammars in an artificial language [Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1999). Science, 283, 77–80], and generalizes to examples both outside and inside of the range of training sentences. Knowledge-representation analyses confirm that these networks discover that duplicate words in the sentences are nearly identical and (...) that they use this near-identity relation to distinguish sentences that are consistent or inconsistent with a familiar grammar. Recent simulations that were claimed to show that this model did not really learn these grammars [Vilcu, M., & Hadley, R. F. (2005). Minds and Machines, 15, 359–382] confounded syntactic types with speech sounds and did not perform standard statistical tests of results. (shrink)