This paper proposes several recurrent neural network-based models for recognizing requisite and effectuation parts in Legal Texts. Firstly, we propose a modification of BiLSTM-CRF model that allows the use of external features to improve the performance of deep learning models in case large annotated corpora are not available. However, this model can only recognize RE parts which are not overlapped. Secondly, we propose two approaches for recognizing overlapping RE parts including the cascading approach which uses the sequence of (...) BiLSTM-CRF models and the unified model approach with the multilayer BiLSTM-CRF model and the multilayer BiLSTM-MLP-CRF model. Experimental results on two Japan law RRE datasets demonstrated advantages of our proposed models. For the Japanese National Pension Law dataset, our approaches obtained an \ score of 93.27% and exhibited a significant improvement compared to previous approaches. For the Japan Civil Code RRE dataset which is written in English, our approaches produced an \ score of 78.24% in recognizing RE parts that exhibited a significant improvement over strong baselines. In addition, using external features and in-domain pre-trained word embeddings also improved the performance of RRE systems. (shrink)

There is increasing interest in how the pupil dynamics of the eye reflect underlying cognitive processes and brain states. Problematic, however, is that pupil changes can be due to non-cognitive factors, for example luminance changes in the environment, accommodation and movement. In this paper we consider how by modeling the response of the pupil in real-world environments we can capture the non-cognitive related changes and remove these to extract a residual signal which is a better index of cognition and performance. (...) Specifically, we utilize sequence measures such as fixation position, duration, saccades, and blink-related information as inputs to a deep recurrent neural network (RNN) model for predicting subsequent pupil diameter. We build and evaluate the model for a task where subjects are watching educational videos and subsequently asked questions based on the content. Compared to commonly-used models for this task, the RNN had the lowest errors rates in predicting subsequent pupil dilation given sequence data. Most importantly was how the model output related to subjects' cognitive performance as assessed by a post-viewing test. Consistent with our hypothesis that the model captures non-cognitive pupil dynamics, we found (1) the model's root-mean square error was less for lower performing subjects than for those having better performance on the post-viewing test, (2) the residuals of the RNN (LSTM) model had the highest correlation with subject post-viewing test scores and (3) the residuals had the highest discriminability (assessed via area under the ROC curve, AUC) for classifying high and low test performers, compared to the true pupil size or the RNN model predictions. This suggests that deep learning sequence models may be good for separating components of pupil responses that are linked to luminance and accommodation from those that are linked to cognition and arousal. (shrink)

Timely detection and efficient recognition of fault are challenging for the bogie of high-speed train, owing to the fact that different types of fault signals have similar characteristics in the same frequency range. Notice that convolutional neural networks are powerful in extracting high-level local features and that recurrent neural networks are capable of learning long-term context dependencies in vibration signals. In this paper, by combining CNN and RNN, a so-called convolutional recurrent neural network (...) is proposed to diagnose various faults of the HST bogie, where the capabilities of CNN and RNN are inherited simultaneously. Within the novel architecture, the proposed CRNN first filters out the features from the original data through convolutional layers. Then, four recurrent layers with simple recurrent cell are used to model the context information in the extracted features. By comparing the performance of the presented CRNN with CNN, RNN, and ensemble learning, experimental results show that CRNN achieves not only the best performance with accuracy of 97.8% but also the least time spent in training model. (shrink)

It has been shown that Recurrent Artificial Neural Networks automatically acquire some grammatical knowledge in the course of performing linguistic prediction tasks. The extent to which such networks can actually learn grammar is still an object of investigation. However, being mostly data-driven, they provide a natural testbed for usage-based theories of language acquisition. This mini-review gives an overview of the state of the field, focusing on the influence of the theoretical framework in the interpretation of results.

It was meaningful to predict the customers' decision-making behavior in the field of market. However, due to individual differences and complex, non-linear natures of the electroencephalogram signals, it was hard to classify the EEG signals and to predict customers' decisions by using traditional classification methods. To solve the aforementioned problems, a recurrent t-distributed stochastic neighbor embedding neural network was proposed in current study to classify the EEG signals in the designed brand extension paradigm and to predict the participants' (...) decisions. The recurrent t-SNE neural network contained two steps. In the first step, t-SNE algorithm was performed to extract features from EEG signals. Second, a recurrent neural network with long short-term memory layer, fully connected layer, and SoftMax layer was established to train the features, classify the EEG signals, as well as predict the cognitive performance. The proposed network could give a good prediction with accuracy around 87%. Its superior in prediction accuracy as compared to a recurrent principal component analysis network, a recurrent independent component correlation algorithm [independent component analysis ] network, a t-SNE support vector machine network, a t-SNE back propagation neural network, a deep LSTM neural network, and a convolutional neural network were also demonstrated. Moreover, the performance of the proposed network with different activated channels were also investigated and compared. The results showed that the proposed network could make a relatively good prediction with only 16 channels. The proposed network would become a potentially useful tool to help a company in making marketing decisions and to help uncover the neural mechanisms behind individuals' decision-making behavior with low cost and high efficiency. (shrink)

Day-to-day lives are affected globally by the epidemic coronavirus 2019. With an increasing number of positive cases, India has now become a highly affected country. Chronic diseases affect individuals with no time identification and impose a huge disease burden on society. In this article, an Efficient Recurrent Neural Network with Ensemble Classifier is built using VGG-16 and Alexnet with weighted model to predict disease and its level. The dataset is partitioned randomly into small subsets by utilizing mean-based splitting (...) method. Various models of classifier create a homogeneous ensemble by utilizing an accuracy-based weighted aging classifier ensemble, which is a weighted model’s modification. Two state of art methods such as Graph Sequence Recurrent Neural Network and Hybrid Rough-Block-Based Neural Network are used for comparison with respect to some parameters such as accuracy, precision, recall, f1-score, and relative absolute error. As a result, it is found that the proposed ERNN-EC method accomplishes accuracy of 95.2%, precision of 91%, recall of 85%, F1-score of 83.4%, and RAE of 41.6%. (shrink)

Recent decades have witnessed the rapid evolution of robotic applications and their expansion into a variety of spheres with remarkable achievements. This article researches a crucial technique of robot manipulators referred to as visual servoing, which relies on the visual feedback to respond to the external information. In this regard, the visual servoing issue is tactfully transformed into a quadratic programming problem with equality and inequality constraints. Differing from the traditional methods, a gradient-based recurrent neural network for solving (...) the visual servoing issue is newly proposed in this article in the light of the gradient descent method. Then, the stability proof is presented in theory with the pixel error convergent exponentially to zero. Specifically speaking, the proposed method is able to impel the manipulator to approach the desired static point while maintaining physical constraints considered. After that, the feasibility and superiority of the proposed GRNN are verified by simulative experiments. Significantly, the proposed visual servo method can be leveraged to medical robots and rehabilitation robots to further assist doctors in treating patients remotely. (shrink)

This paper implements the continuous Hindi Automatic Speech Recognition (ASR) system using the proposed integrated features vector with Recurrent Neural Network (RNN) based Language Modeling (LM). The proposed system also implements the speaker adaptation using Maximum-Likelihood Linear Regression (MLLR) and Constrained Maximum likelihood Linear Regression (C-MLLR). This system is discriminatively trained by Maximum Mutual Information (MMI) and Minimum Phone Error (MPE) techniques with 256 Gaussian mixture per Hidden Markov Model(HMM) state. The training of the baseline system has been (...) done using a phonetically rich Hindi dataset. The results show that discriminative training enhances the baseline system performance by up to 3%. Further improvement of ~7% has been recorded by applying RNN LM. The proposed Hindi ASR system shows significant performance improvement over other current state-of-the-art techniques. (shrink)

Paul Feyerabend recommended the methodological policy of proliferating competing theories as a means to uncovering new empirical data, and thus as a means to increase the empirical constraints that all theories must confront. Feyerabend's policy is here defended as a clear consequence of connectionist models of explanatory understanding and learning. An earlier connectionist "vindication" is criticized, and a more realistic and penetrating account is offered in terms of the computationally plastic cognitive profile displayed by neural networks with a (...) recurrent architecture. (shrink)

This article offers a critical inquiry of contemporary neural network models as an instance of machine learning, from an interdisciplinary perspective of AI studies and performativity. It shows the limits on the architecture of these network systems due to the misemployment of ‘natural’ performance, and it offers ‘context’ as a variable from a performative approach, instead of a constant. The article begins with a brief review of machine learning-based natural language processing systems and continues with a concentration on the (...) relevant model of recurrent neural networks, which is applied in most commercial research such as Facebook AI Research. It demonstrates that the logic of performativity is not brought into account in all recurrent nets, which is an integral part of human performance and languaging, and it argues that recurrent network models, in particular, fail to grasp human performativity. This logic works similarly to the theory of performativity articulated by Jacques Derrida in his critique of John L. Austin’s concept of the performative. Applying Jacques Derrida’s work on performativity, and linguistic traces as spatially organized entities that allow for this notion of performance, the article argues that recurrent nets fall into the trap of taking ‘context’ as a constant, of treating human performance as a ‘natural’ fix to be encoded, instead of performative. Lastly, the article applies its proposal more concretely to the case of Facebook AI Research’s Alice and Bob. (shrink)

This paper focuses on the finite-time synchronization analysis for complex-valued recurrent neural networks with time delays. First, two kinds of common activation functions appearing in the existing references are combined together and more general assumptions are given. To achieve our aim, a nonlinear delayed controller with two independent parameters different from the existing ones is provided, which leads to great difficulty. To overcome it, a newly developed inequality is used. Then, via Lyapunov function approach, some criteria are (...) derived to guarantee the finite-time synchronization of the considered system, and the settling time for synchronization is also estimated. Finally, two numerical simulations are given to support the effectiveness and advantages of the obtained results. (shrink)

Hate speech has spread more rapidly through the daily use of technology and, most notably, by sharing your opinions or feelings on social media in a negative aspect. Although numerous works have been carried out in detecting hate speeches in English, German, and other languages, very few works have been carried out in the context of the Bengali language. In contrast, millions of people communicate on social media in Bengali. The few existing works that have been carried out need improvements (...) in both accuracy and interpretability. This article proposed encoder–decoder-based machine learning model, a popular tool in NLP, to classify user’s Bengali comments from Facebook pages. A dataset of 7,425 Bengali comments, consisting of seven distinct categories of hate speeches, was used to train and evaluate our model. For extracting and encoding local features from the comments, 1D convolutional layers were used. Finally, the attention mechanism, LSTM, and GRU-based decoders have been used for predicting hate speech categories. Among the three encoder–decoder algorithms, attention-based decoder obtained the best accuracy (77%). (shrink)

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine, a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can (...) encode contextual information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models. We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain. (shrink)

Current cognitive science models of perception and action assume that the objects that we move toward and perceive are represented as determinate in our experience of them. A proper phenomenology of perception and action, however, shows that we experience objects indeterminately when we are perceiving them or moving toward them. This indeterminacy, as it relates to simple movement and perception, is captured in the proposed phenomenologically based recurrent network models of brain function. These models provide a possible foundation from (...) which predicative structures may arise as an emergent phenomenon without the positing of a representing subject. These models go some way in addressing the dual constraints of phenomenological accuracy and neurophysiological plausibility that ought to guide all projects devoted to discovering the physical basis of human experience. (shrink)

A session-based recommendation system is designed to predict the user’s next click behavior based on an ongoing session. Existing session-based recommendation systems usually model a session into a sequence and extract sequence features through recurrent neural network. Although the performance is greatly improved, these procedures ignore the relationships between items that contain rich information. In order to obtain rich items embeddings, we propose a novel Recommendation Model based on Multi-channel Convolutional Neural Network for session-based recommendation, RMMCNN for (...) brevity. Specifically, we capture items' internal features from three dimensions through multi-channel convolutional neural network firstly. Next, we merge the internal features with external features obtained by a GRU unit. Then, both internal features and external features are merged by an attention mechanism together as the input of the transformation function. Finally, the probability distribution is taken as the output after the softmax function. Experiments on various datasets show that our method's precision and recommendation performance are better than those of other state-of-the-art approaches. (shrink)

Recent work on the application of neural networks to language modeling has shown that models based on certain neural architectures can capture syntactic information from utterances and sentences even when not given an explicitly syntactic objective. We examine whether a fully data-driven model of language development that uses a recurrent neural network encoder for utterances can track how child language utterances change over the course of language development in a way that is comparable to what (...) is achieved using established language assessment metrics that use language-specific information carefully designed by experts. Given only transcripts of child language utterances from the CHILDES Database and no pre-specified information about language, our model captures not just the structural characteristics of child language utterances, but how these structures reflect language development over time. We establish an evaluation methodology with which we can examine how well our model tracks language development compared to three known approaches: Mean Length of Utterance, the Developmental Sentence Score, and the Index of Productive Syntax. We discuss the applicability of our model to data-driven assessment of child language development, including how a fully data-driven approach supports the possibility of increased research in multilingual and cross-lingual issues. (shrink)

In today’s era of rapid development in science and technology, the development of digital technology has increasingly higher requirements for data processing functions. The matrix signal commonly used in engineering applications also puts forward higher requirements for processing speed. The eigenvalues of the matrix represent many characteristics of the matrix. Its mathematical meaning represents the expansion of the inherent vector, and its physical meaning represents the spectrum of vibration. The eigenvalue of a matrix is the focus of matrix theory. The (...) problem of matrix eigenvalues is widely used in many research fields such as physics, chemistry, and biology. A neural network is a neuron model constructed by imitating biological neural networks. Since it was proposed, the application research of its typical models, such as recurrent neural networks and cellular neural networks, has become a new hot spot. With the emergence of deep neural network theory, scholars continue to combine deep neural networks to calculate matrix eigenvalues. This article aims to study the estimation and application of matrix eigenvalues based on deep neural networks. This article introduces the related methods of matrix eigenvalue estimation based on deep neural networks, and also designs experiments to compare the time of matrix eigenvalue estimation methods based on deep neural networks and traditional algorithms. It was found that under the serial algorithm, the algorithm based on the deep neural network reduced the calculation time by about 7% compared with the traditional algorithm, and under the parallel algorithm, the calculation time was reduced by about 17%. Experiments are also designed to calculate matrix eigenvalues with Obj and recurrent neural networks (RNNS) models, which proves that the Oja algorithm is only suitable for calculating the maximum eigenvalues of non-negative matrices, while RNNS is commonly used in general models. (shrink)

Theoretical research into the emotional attributes of ideological and political education can improve our ability to understand human emotion and solve socio-emotional problems. To that end, this study undertook an analysis of emotion in ideological and political education by integrating a gate recurrent unit with an attention mechanism. Based on the good results achieved by BERT in the downstream network, we use the long focusing attention mechanism assisted by two-way GRU to extract relevant information and global information of ideological (...) and political education and emotion analysis, respectively. The two kinds of information complement each other, and the accuracy of emotion information can be further improved by combining neural network model. Finally, the validity and domain adaptability of the model were verified using several publicly available, fine-grained emotion datasets. (shrink)

Chaos in nervous system is a fascinating but controversial field of investigation. To approach the role of chaos in the real brain, we theoretically and numerically investigate the occurrence of chaos inartificial neural networks. Most of the time, recurrent networks (with feedbacks) are fully connected. This architecture being not biologically plausible, the occurrence of chaos is studied here for a randomly diluted architecture. By normalizing the variance of synaptic weights, we produce a bifurcation parameter, dependent on (...) this variance and on the slope of the transfer function, that allows a sustained activity and the occurrence of chaos when reaching a critical value. Even for weak connectivity and small size, we find numerical results in accordance with the theoretical ones previously established for fully connected infinite sized networks. The route towards chaos is numerically checked to be a quasi-periodic one, whatever the type of the first bifurcation is. Our results suggest that such high-dimensional networks behave like low-dimensional dynamical systems. (shrink)

With a focus in the financial market, stock market dynamics forecasting has received much attention. Predicting stock market fluctuations is usually challenging due to the nonlinear and nonstationary time series of stock prices. The Elman recurrent network is renowned for its capability of dealing with dynamic information, which has made it a successful application to predicting. We developed a hybrid approach which combined Elman recurrent network with factorization machine technique, i.e., the FM-Elman neural network, to predict stock (...) market volatility. In this paper, the Standard & Poor’s 500 Composite Stock Price index, the Dow Jones industrial average index, the Shanghai Stock Exchange Composite index, and the Shenzhen Securities Component Index were used to demonstrate the validity of our proposed FM-Elman model in time-series prediction. The results were compared with predictions obtained from the other two models which are basic BP neural network and the Elman neural network. Some experiments showed that the FM-Elman model outperforms others through different accuracy measures. Furthermore, the effects of volatility degree on prediction performance from different stock indexes were investigated. An interesting phenomenon had been found through some numerical experiments on the effects of different user-specified dimensions on the proposed FM-Elman neural network. (shrink)

. Fuzzy systems and artificial neural networks, as important components of soft-computation, can be applied together to model uncertainty. A composite block of the fuzzy system and the ANN shares a mutually beneficial association resulting in enhanced performance with smaller networks. It makes them suitable for application with time-varying multi-input multi-output channel modeling enabling such a system to track minute variations in propagation conditions. Here we propose a fuzzy neural system using a fuzzy time delay fully (...) recurrent neural network that has the capability to tackle time-varying inputs in fuzzified form and is used to model MIMO channels. The inference engine is constituted by novel FTDFRNN blocks which determine the decision boundaries and tracks in-phase and quadrature components of input signals encompassing stochastic behavior of the MIMO channel. The system shows significant improvement in performance compared to statistical and ANN approaches in terms of faster processing time, lower bit error rate margins and better precision while carrying out symbol recovery of transmitted data through severely faded MIMO channels. (shrink)

Sibley et al. (2008) report a recurrent neural network model designed to learn wordform representations suitable for written and spoken word identification. The authors claim that their sequence encoder network overcomes a key limitation associated with models that code letters by position (e.g., CAT might be coded as C‐in‐position‐1, A‐in‐position‐2, T‐in‐position‐3). The problem with coding letters by position (slot‐coding) is that it is difficult to generalize knowledge across positions; for example, the overlap between CAT and TOMCAT is lost. (...) Although we agree this is a critical problem with many slot‐coding schemes, we question whether the sequence encoder model addresses this limitation, and we highlight another deficiency of the model. We conclude that alternative theories are more promising. (shrink)

Recurrent neural network has been widely applied to many sequential tagging tasks such as natural language process and time series analysis, and it has been proved that RNN works well in those areas. In this paper, we propose using RNN with long short-term memory units for server load and performance prediction. Classical methods for performance prediction focus on building relation between performance and time domain, which makes a lot of unrealistic hypotheses. Our model is built based on events, (...) which is the root cause of server performance. We predict the performance of the servers using RNN-LSTM by analyzing the log of servers in data center which contains user’s access sequence. Previous work for workload prediction could not generate detailed simulated workload, which is useful in testing the working condition of servers. Our method provides a new way to reproduce user request sequence to solve this problem by using RNN-LSTM. Experiment result shows that our models get a good performance in generating load and predicting performance on the data set which has been logged in online service. We did experiments with nginx web server and mysql database server, and our methods can been easily applied to other servers in data center. (shrink)

Paul Churchland does not open his latest book,The Engine of Reason, the Seat of the Soul, modestly. He begins by announcing, “This book is about you. And me … More broadly still, it is about every creature that ever swam, or walked, or flew over the face of the Earth” (p. 3). A few sentences later, he says, “Fortunately, recent research into neural networks … has produced the beginnings of a real understanding of how the biological brain works—a (...) real understanding, that is, of howyouwork, and everyone else like you” (p. 3). The implicit identification here of “me and you and everyone” with “the biological brain” might lead an uncharitable reader to view Churchland's book as “Eliminativism for the non-specialist,” that is, as an attempt to popularize the view of the mindbody problem with which, among his professional peers, Churchland has long been identified. However, I think that such a readingwouldbe uncharitable. He is, of course, frequently sceptical about the utility of folk psychology, but in this book he is much less concerned to disparage folk psychology as a failedtheory(by contrast with, for example, the arguments in Churchland 1979) than to urge the more modest view that the more we understand the brain, the better we shall be at helping those whose brains are damaged in ways that interfere seriously with the fulfilment of their lives. Hence, I am inclined to take him at his word when he says in the Preface that “The book is motivated first of all by sheer excitement over the new picture that is now emerging … [and] … also by the idea that this is information that the public needs to know” (p. xi). What excites Churchland so, at least overtly, is not the negative thesis he has defended elsewhere that folk-psychological terms fail to refer; his enthusiasm is mainly reserved for the positive thesis that minds are, essentially, interacting assemblies of recurrent neural networks. It is therefore this positive thesis, and Churchland's defence of it, that I will assess in the following discussion. (shrink)

Introduction of deep neural networks to the machine translation research ameliorated conventional machine translation systems in multiple ways, specifically in terms of translation quality. The ability of deep neural networks to learn a sensible representation of words is one of the major reasons for this improvement. Despite machine translation using deep neural architecture is showing state-of-the-art results in translating European languages, we cannot directly apply these algorithms in Indian languages mainly because of two reasons: unavailability (...) of the good corpus and Indian languages are morphologically rich. In this paper, we propose a neural machine translation (NMT) system for four language pairs: English–Malayalam, English–Hindi, English–Tamil, and English–Punjabi. We also collected sentences from different sources and cleaned them to make four parallel corpora for each of the language pairs, and then used them to model the translation system. The encoder network in the NMT architecture was designed with long short-term memory (LSTM) networks and bi-directional recurrent neural networks (Bi-RNN). Evaluation of the obtained models was performed both automatically and manually. For automatic evaluation, the bilingual evaluation understudy (BLEU) score was used, and for manual evaluation, three metrics such as adequacy, fluency, and overall ranking were used. Analysis of the results showed the presence of lengthy sentences in English–Malayalam, and the English–Hindi corpus affected the translation. Attention mechanism was employed with a view to addressing the problem of translating lengthy sentences (sentences contain more than 50 words), and the system was able to perceive long-term contexts in the sentences. (shrink)

The dynamics of neural populations capture the laws of the mindThis paper focuses on the level of neural networks. Examining the case of recurrent neural networks, the paper argues that the dynamics of neural populations form a privileged level of explanation in cognitive science. According to Schöner, this level is privileged, because it enables cognitive scientists to discover the laws governing organisms’ cognition and behaviour.

One challenge in explaining neural evolution is the formal equivalence of different computational architectures. If a simple architecture suffices, why should more complex neural architectures evolve? The answer must involve the intense competition for resources under which brains operate. I show how recurrent neural networks can be favored when increased complexity allows for more efficient use of existing resources. Although resource constraints alone can drive a change, recurrence shifts the landscape of what is later evolvable. (...) Hence organisms on either side of a transition boundary may have similar cognitive capacities but very different potential for evolving new capacities. (shrink)

It is crucial, first of all, to stress the importance Churchland attaches to the idea that the neural networks whose assemblages he holds to be “engines of reason” must be recurrent. Non-recurrent networks, of the sort best known among philosophers, simply discover patterns in input data presented to them as sets of features. The learning capacities of such networks, extensively discussed since the publication of Rumelhart and McClelland et al., are indeed impressive; and Churchland (...) describes them clearly and gracefully as preparation for introducing recurrent networks. Now, the importance of recurrence as a feature of the networks Churchland hypothesizes as forming the basis of cognitive activity is well motivated. A cognitive system that was an assembly of non-recurrent networks would be, in essence, a stimulus-response machine of the sort that early behaviourists took themselves to be studying when they examined the mind/brain. Such a system could learn to find novel patterns in data. However, it is empirically evident that human— and some non-human—cognitive capacities go well beyond this. People dream, imagine scenarios that never existed, reconceptualize their perceptions, and theorize. All these activities seem to require not merely inferring patterns from data, but reconceiving the nature of the data itself in light of the inferential structures built through learning. For this to be possible, the data cannot be “clamped,” as they are in the case of non-recurrent networks. Hence the importance of recurrence. A recurrent network can be defined as one which takes some or all of its own output and then treats that output as a source of input. This allows for the recognition of meta-patterns. Furthermore, if some such networks are feeding their output as input to other networks with which they are linked, the possibility of reasoning and conceptualizing by analogy arises; and this ability, as Churchland argues, seems to be an essential aspect of genuine creativity in science, art, and the continuously ongoing re-design of the social order. The actual existence of simulated recurrent networks, and the capacities they demonstrate, constitute an impressive possibility argument that a system with enormous cognitive plasticity and behavioural flexibility could be built out of parallel distributed processing networks, so long as at least many of these networks were recurrent. The importance of this possibility argument should not be under-emphasized, but it of course invites an obvious question: how good are our grounds for supposing that the possibility in question is actually realized in the biological domain? (shrink)

Accurate solar irradiance prediction plays an important role in ensuring the security and stability of renewable energy systems. Solar irradiance modeling is usually a time-dependent dynamic model. As a new kind of recurrent neural network, echo state network shows excellent performance in the field of time series prediction. However, the memory length of classical ESN is fixed and finite, which makes it hard to map sufficient features of solar irradiance with long-range dependency. Therefore, a novel deep echo state (...) network with variable memory pattern is proposed in this brief. VMP-DESN consists of multiple connected reservoirs in series, and there exist different types of memory modes in VMP-DESN. To remember more input history information in the states, the time delay links are added in each reservoir and between every two reservoirs. The VMP-DESN is more flexible to deal with different input signals due to its variable memory modes in the reservoir states. Additionally, the effect of different memory patterns on the VMP-DESN performance is discussed in detail, including the antidisturbance ability, memory capacity, and prediction accuracy. Finally, the effectiveness of VMP-DESN is evaluated by predicting the real solar irradiance task. (shrink)

The entropy-reduction hypothesis claims that the cognitive processing difficulty on a word in sentence context is determined by the word's effect on the uncertainty about the sentence. Here, this hypothesis is tested more thoroughly than has been done before, using a recurrent neural network for estimating entropy and self-paced reading for obtaining measures of cognitive processing load. Results show a positive relation between reading time on a word and the reduction in entropy due to processing that word, supporting (...) the entropy-reduction hypothesis. Although this effect is independent from the effect of word surprisal, we find no evidence that these two measures correspond to cognitively distinct processes. (shrink)