I address whether neural networks perform computations in the sense of computability theory and computer science. I explicate and defend
the following theses. (1) Many neural networks compute—they perform computations. (2) Some neural networks compute in a classical way.
Ordinary digital computers, which are very large networks of logic gates, belong in this class of neural networks. (3) Other neural networks
compute in a non-classical way. (4) Yet other neural (...) class='Hi'>networks do not perform computations. Brains may well fall into this last class. (shrink)

An ideal vision model accounts for behavior and neurophysiology in both naturalistic conditions and designed lab experiments. Unlike psychological theories, artificial neural networks (ANNs) actually perform visual tasks and generate testable predictions for arbitrary inputs. These advantages enable ANNs to engage the entire spectrum of the evidence. Failures of particular models drive progress in a vibrant ANN research program of human vision.

The computer distance teaching system teaches through the network, and there is no entrance threshold. Any student who is willing to study can log in to the network computer distance teaching system for study at any free time. Neural network has a strong self-learning ability and is an important part of artificial intelligence research. Based on this study, a neural network-embedded architecture based on shared memory and bus structure is proposed. By looking for an alternative method (...) of exp function to improve the speed of radial basis function algorithm, and then by analyzing the judgment conditions in the main loop during the algorithm process, these judgment conditions are modified conditionally to reduce the calculation scale, which can double the speed of the algorithm. Finally, this article verifies the function, performance, and interface of the computer distance education system. (shrink)

Adaptive dynamic programming, which belongs to the field of computational intelligence, is a powerful tool to address optimal control problems. To overcome the bottleneck of solving Hamilton–Jacobi–Bellman equations, several state-of-the-art ADP approaches are reviewed in this paper. First, two model-based offline iterative ADP methods including policy iteration and value iteration are given, and their respective advantages and shortcomings are discussed in detail. Second, the multistep heuristic dynamic programming method is introduced, which avoids the requirement of initial admissible control and achieves (...) fast convergence. This method successfully utilizes the advantages of PI and VI and overcomes their drawbacks at the same time. Finally, the discrete-time optimal control strategy is tested on a power system. (shrink)

Fluid neural networks can be used as a theoretical framework for a wide range of complex systems as social insects. In this article we show that collective logical gates can be built in such a way that complex computation can be possible by means of the interplay between local interactions and the collective creation of a global field. This is exemplified by a NOR gate. Some general implications for ant societies are outlined. ©.

A basic deep neural network (DNN) is trained to exhibit a large set of input–output dispositions. While being a good model of the way humans perform some tasks automatically, without deliberative reasoning, more is needed to approach human‐like artificial intelligence. Analysing recent additions brings to light a distinction between two fundamentally different styles of computation: content‐specific and non‐content‐specific computation (as first defined here). For example, deep episodic RL networks draw on both. So does human conceptual reasoning. Combining the (...) two takes advantage of the complementary costs and benefits of each. It also offers a better model of human cognitive competence. (shrink)

There is a distinction between locality and modularity. These two terms have often been used interchangeably in the target article and commentary. Using this distinction we argue in favor of a modularity. In addition we also argue that both PDP-type networks and box-and-arrow models have their own strengths and pitfalls.

Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images of objects and are often described as the best models of biological vision. This conclusion is largely based on three sets of findings: (1) DNNs are more accurate than any other model in classifying images taken from various datasets, (2) DNNs do the best job in predicting the pattern of human errors in classifying objects taken from various behavioral datasets, and (3) DNNs do the best job (...) in predicting brain signals in response to images taken from various brain datasets (e.g., single cell responses or fMRI data). However, these behavioral and brain datasets do not test hypotheses regarding what features are contributing to good predictions and we show that the predictions may be mediated by DNNs that share little overlap with biological vision. More problematically, we show that DNNs account for almost no results from psychological research. This contradicts the common claim that DNNs are good, let alone the best, models of human object recognition. We argue that theorists interested in developing biologically plausible models of human vision need to direct their attention to explaining psychological findings. More generally, theorists need to build models that explain the results of experiments that manipulate independent variables designed to test hypotheses rather than compete on making the best predictions. We conclude by briefly summarizing various promising modeling approaches that focus on psychological data. (shrink)

In step with rapid advancements in computer vision, vehicle classification demonstrates a considerable potential to reshape intelligent transportation systems. In the last couple of decades, image processing and pattern recognition-based vehicle classification systems have been used to improve the effectiveness of automated highway toll collection and traffic monitoring systems. However, these methods are trained on limited handcrafted features extracted from small datasets, which do not cater the real-time road traffic conditions. Deep learning-based classification systems have been proposed to incorporate (...) the above-mentioned issues in traditional methods. However, convolutional neural networks require piles of data including noise, weather, and illumination factors to ensure robustness in real-time applications. Moreover, there is no generalized dataset available to validate the efficacy of vehicle classification systems. To overcome these issues, we propose a convolutional neural network-based vehicle classification system to improve robustness of vehicle classification in real-time applications. We present a vehicle dataset comprising of 10,000 images categorized into six-common vehicle classes considering adverse illuminous conditions to achieve robustness in real-time vehicle classification systems. Initially, pretrained AlexNet, GoogleNet, Inception-v3, VGG, and ResNet are fine-tuned on self-constructed vehicle dataset to evaluate their performance in terms of accuracy and convergence. Based on better performance, ResNet architecture is further improved by adding a new classification block in the network. To ensure generalization, we fine-tuned the network on the public VeRi dataset containing 50,000 images, which have been categorized into six vehicle classes. Finally, a comparison study has been carried out between the proposed and existing vehicle classification methods to evaluate the effectiveness of the proposed vehicle classification system. Consequently, our proposed system achieved 99.68%, 99.65%, and 99.56% accuracy, precision, and F1-score on our self-constructed dataset. (shrink)

The purpose is to improve the teaching and learning efficiency of college Innovation and Entrepreneurship Education. Firstly, from the perspective of aesthetic education, this work designs the teacher and student sides of the Computer-aided Instruction system. Secondly, the CAI model is implemented based on the weight sharing and local perception of the Convolutional Neural Network. Finally, the performance of the CNN-based CAI model is tested. Meanwhile, it analyses students’ IEE experience under the proposed CAI model through a case (...) study of Music Majors from Xi’an Conservatory of Music. The experimental data show that the CNN-based CAI model can respond quickly and stably when users access different functional modules, such as webpage browsing. The proposed CAI model increases students’ entrepreneurial interest, skills, and knowledge by 55.62, 57.32, and 72.12%, respectively. Students’ entrepreneurial practice ability has been improved by over 50.00%; such an increase in entrepreneurial practice ability has also shown individual differences. Thus, the proposed Music Majors-oriented IEE-infiltrated CAI model based on CNN improves students’ entrepreneurial practice ability and reflects the positive experience of Music Majors on IEE. The finding provides references for the step-by-step identification of the CNN-based CAI model and has certain guiding significance for analyzing the effect of college IEE. (shrink)

Computer simulations show that an unstructured neural-network model [Shultz, T. R., & Bale, A. C. (2001). Infancy, 2, 501–536] covers the essential features␣of infant learning of simple grammars in an artificial language [Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1999). Science, 283, 77–80], and generalizes to examples both outside and inside of the range of training sentences. Knowledge-representation analyses confirm that these networks discover that duplicate words in the sentences are nearly identical (...) and that they use this near-identity relation to distinguish sentences that are consistent or inconsistent with a familiar grammar. Recent simulations that were claimed to show that this model did not really learn these grammars [Vilcu, M., & Hadley, R. F. (2005). Minds and Machines, 15, 359–382] confounded syntactic types with speech sounds and did not perform standard statistical tests of results. (shrink)

In order to benefit from the advantages of localist coding, neural models that feature winner-take-all representations at the top level of a network hierarchy must still solve the computational problems inherent in distributed representations at the lower levels.

Cultural similarities and differences in facial expressions have been a controversial issue in the field of facial communications. A key step in addressing the debate regarding the cultural dependency of emotional expression is to characterize the visual features of specific facial expressions in individual cultures. Here we developed an image analysis framework for this purpose using convolutional neural networks that through training learned visual features critical for classification. We analyzed photographs of facial expressions derived from two databases, each (...) developed in a different country, in which corresponding emotion labels were available. While the CNNs reached high rates of correct results that were far above chance after training with each database, they showed many misclassifications when they analyzed faces from the database that was not used for training. These results suggest that facial features useful for classifying facial expressions differed between the databases. The selectivity of computational units in the CNNs to action units of the face varied across the facial expressions. Importantly, the AU selectivity often differed drastically between the CNNs trained with the different databases. Similarity and dissimilarity of these tuning profiles partly explained the pattern of misclassifications, suggesting that the AUs are important for characterizing the facial features and differ between the two countries. The AU tuning profiles, especially those reduced by principal component analysis, are compact summaries useful for comparisons across different databases, and thus might advance our understanding of universality vs. specificity of facial expressions across cultures. (shrink)

This work describes a mathematical concept of modelling field theory and its applications to a variety of problems, while offering a view of the relationships among mathematics, computational concepts in neural networks, semiotics, and concepts of mind in psychology and philosophy.

Emotion recognition plays an important part in human-computer interaction. Currently, the main challenge in electroencephalogram -based emotion recognition is the non-stationarity of EEG signals, which causes performance of the trained model decreasing over time. In this paper, we propose a two-level domain adaptation neural network to construct a transfer model for EEG-based emotion recognition. Specifically, deep features from the topological graph, which preserve topological information from EEG signals, are extracted using a deep neural network. These features are (...) then passed through TDANN for two-level domain confusion. The first level uses the maximum mean discrepancy to reduce the distribution discrepancy of deep features between source domain and target domain, and the second uses the domain adversarial neural network to force the deep features closer to their corresponding class centers. We evaluated the domain-transfer performance of the model on both our self-built data set and the public data set SEED. In the cross-day transfer experiment, the ability to accurately discriminate joy from other emotions was high: sadness, anger, and fear on the self-built data set. The accuracy reached 74.93% on the SEED data set. In the cross-subject transfer experiment, the ability to accurately discriminate joy from other emotions was equally high: sadness, anger, and fear on the self-built data set. The average accuracy reached 87.9% on the SEED data set, which was higher than WGAN-DA. The experimental results demonstrate that the proposed TDANN can effectively handle the domain transfer problem in EEG-based emotion recognition. (shrink)

Cook (1995) criticizes the work of Jacobs and Kosslyn (1994) on spatial relations, shape representations, and receptive fields in neural network models on the grounds that first‐order correlations between input and output unit activities can explain the results. We reply briefly to Cook's arguments here (and in Kosslyn, Chabris, Marsolek, Jacobs & Koenig, 1995) and discuss how new simulations can confirm the importance of receptive field size as a crucial variable in the encoding of categorical and coordinate spatial relations (...) and the corresponding shape representations; such simulations would testify to the computational distinction between the different types of representations. (shrink)

Court rulings are among the most important documents in all legal systems. This article describes a study in which natural language processing is used for the automatic characterization of Spanish judgments that deal with the physical custody (joint or individual) of minors. The model was trained to identify a set of elements: the type of custody requested by the plaintiff, the type of custody decided on by the court, and eight of the most commonly used arguments in this type of (...) judgment. Two jurists independently annotated more than 3000 judgments, which were used to train a model based on transformers. The main difficulties encountered in this task were the complexity of the judicial language and the need to work with appellate court rulings that have a more complicated structure than decisions at first instance. For the complete court rulings, the F1 score of the inter-annotator agreement ranged from 0.60 to 0.86 and the Kappa index from 0.33 to 0.73. The F1 score of the agreement between the model and the annotators ranged from 0.66 to 0.93 and the Kappa index from 0.57 to 0.80. These results in which the model performance exceeds even the inter-annotator agreement show the high ability of transformers to identify abstract entities in legal texts. (shrink)

With the continual development of artificial intelligence and smart computing in recent years, quantitative approaches have become increasingly popular as an efficient modeling tool as they do not necessitate complicated mathematical models. Many nations have taken steps, such as transitioning to online schooling, to decrease the harm caused by coronaviruses. Inspired by the demand for technology in early education, the present research uses a radial basis function neural network modeling technique to predict preschool instructors’ technology usage in classes based (...) on recognized determinant characteristics of technology acceptance. In this regard, this study utilized the RBFNN approach to predict preschool teachers’ technology acceptance behavior, based on the theory of planned behavior, which states that behavioral achievement, in our case the actual technology use in class, depends on motivation, intention and ability, and behavioral control. Thus, this research design is based on an adapted version of the technology acceptance model with eight dimensions: D1. Perceived usefulness, D2. Perceived ease of use, D3. Perceived enjoyment, D4. Intention to use, D5. Actual use, D6. Compatibility, D7. Attitude, and D8. Self-efficacy. According to the TAM, actual usage is significantly predicted by the other seven dimensions used in this research. Instead of using the classical multiple linear regression statistical processing of data, we opted for a NN based on the RBF approach to predict the actual usage behavior. This study included 182 preschool teachers who were randomly chosen from a project-based national preschool teacher training program and who responded to our online questionnaire. After designing the RBF function with the actual usage as an output variable and the other seven dimensions as input variables, in the model summary, we obtained in the training sample a sum of squares error of 37.5 and a percent of incorrect predictions of 43.3%. In the testing sample, we obtained a sum of squares error of 14.88 and a percent of incorrect predictions of 37%. Thus, we can conclude that 63% of the classified data are correctly assigned to the models’ dependent variable, i.e., actual technology use, which is a significant rate of correct predictions in the testing sample. This high significant percentage of correct classification represents an important result, mainly because this is the first study to apply RBFNN’s prediction on psychological data, opening up a new interdisciplinary field of research. (shrink)

Functional near-infrared spectroscopy has attracted increasing attention in the field of brain–computer interfaces owing to their advantages such as non-invasiveness, user safety, affordability, and portability. However, fNIRS signals are highly subject-specific and have low test-retest reliability. Therefore, individual calibration sessions need to be employed before each use of fNIRS-based BCI to achieve a sufficiently high performance for practical BCI applications. In this study, we propose a novel deep convolutional neural network -based approach for implementing a subject-independent fNIRS-based BCI. (...) A total of 18 participants performed the fNIRS-based BCI experiments, where the main goal of the experiments was to distinguish a mental arithmetic task from an idle state task. Leave-one-subject-out cross-validation was employed to evaluate the average classification accuracy of the proposed subject-independent fNIRS-based BCI. As a result, the average classification accuracy of the proposed method was reported to be 71.20 ± 8.74%, which was higher than the threshold accuracy for effective BCI communication as well as that obtained using conventional shrinkage linear discriminant analysis. To achieve a classification accuracy comparable to that of the proposed subject-independent fNIRS-based BCI, 24 training trials were necessary for the traditional subject-dependent fNIRS-based BCI. It is expected that our CNN-based approach would reduce the necessity of long-term individual calibration sessions, thereby enhancing the practicality of fNIRS-based BCIs significantly. (shrink)

In recent years, due to the growing demand for computational resources, particularly in cloud computing systems, the data centers’ energy consumption is continually increasing, which directly causes price rise and reductions of resources’ productivity. Although many energy-aware approaches attempt to minimize the consumption of energy, they cannot minimize the violation of service-level agreements at the same time. In this paper, we propose a method using a granular neural network, which is used to model data processing. This method identifies the (...) physical hosts’ workloads before the overflow and can improve energy consumption while also reducing violation of service-level agreements. Unlike the other techniques that use a single criterion, namely, worked on the basis of the history of using the processor, we simultaneously use all the productivity rates criteria, that is, processor productivity rates, main memory, and bandwidth. Extensive real-world simulations using the CloudSim simulator show the high efficiency of the proposed algorithm. (shrink)

How might artificial neural networks (ANNs) inform cognitive science? Often cognitive scientists use ANNs but do not examine their internal structures. In this paper, we use ANNs to explore how cognition might represent musical properties. We train ANNs to classify musical chords, and we interpret network structure to determine what representations ANNs discover and use. We find connection weights between input units and hidden units can be described using Fourier phase spaces, a representation studied in musical set theory. (...) We find the total signal coming through these weighted connection weights is a measure of the similarity between two Fourier structures: the structure of the hidden unit's weights and the structure of the stimulus. This is surprising because neither of these Fourier structures is computed by the hidden unit. We then show how output units use such similarity measures to classify chords. However, we also find different types of units—units that use different activation functions—use this similarity measure very differently. This result, combined with other findings, indicates that while our networks are related to the Fourier analysis of musical sets, they do not perform Fourier analyses of the kind usually described in musical set theory. Our results show Fourier representations of music are not limited to musical set theory. Our results also suggest how cognitive psychologists might explore Fourier representations in musical cognition. Critically, such theoretical and empirical implications require researchers to understand how network structure converts stimuli into responses. (shrink)

The so-called Baldwin Effect generally says how learning, as a form of ontogenetic adaptation, can influence the process of phylogenetic adaptation, or evolution. This idea has also been taken into computation in which evolution and learning are used as computational metaphors, including evolving neural networks. This paper presents a technique called evolving self-taught neural networks – neural networks that can teach themselves without external supervision or reward. The self-taught neural network is intrinsically motivated. (...) Moreover, the self-taught neural network is the product of the interplay between evolution and learning. We simulate a multi-agent system in which neural networks are used to control autonomous agents. These agents have to forage for resources and compete for their own survival. Experimental results show that the interaction between evolution and the ability to teach oneself in self-taught neural networks outperform evolution and self-teaching alone. More specifically, the emergence of an intelligent foraging strategy is also demonstrated through that interaction. Indications for future work on evolving neural networks are also presented. (shrink)

Computer assisted theorem proving is an increasingly important part of mathematical methodology, as well as a long-standing topic in artificial intelligence (AI) research. However, the current generation of theorem proving software have limited functioning in terms of providing new proofs. Importantly, they are not able to discriminate interesting theorems and proofs from trivial ones. In order for computers to develop further in theorem proving, there would need to be a radical change in how the software functions. Recently, machine learning (...) results in solving mathematical tasks have shown early promise that deep artificial neural networks could learn symbolic mathematical processing. In this paper, I analyze the theoretical prospects of such neural networks in proving mathematical theorems. In particular, I focus on the question how such AI systems could be incorporated in practice to theorem proving and what consequences that could have. In the most optimistic scenario, this includes the possibility of autonomous automated theorem provers (AATP). Here I discuss whether such AI systems could, or should, become accepted as active agents in mathematical communities. (shrink)

Augmented reality-based brain–computer interface (AR–BCI) has a low signal-to-noise ratio (SNR) and high real-time requirements. Classical machine learning algorithms that improve the recognition accuracy through multiple averaging significantly affect the information transfer rate (ITR) of the AR–SSVEP system. In this study, a fast recognition method based on a separable convolutional neural network (SepCNN) was developed for an AR-based P300 component (AR–P300). SepCNN achieved single extraction of AR–P300 features and improved the recognition speed. A nine-target AR–P300 single-stimulus paradigm was (...) designed to be administered with AR holographic glasses to verify the effectiveness of SepCNN. Compared with four classical algorithms, SepCNN significantly improved the average target recognition accuracy (81.1%) and information transmission rate (57.90 bits/min) of AR–P300 single extraction. SepCNN with single extraction also attained better results than classical algorithms with multiple averaging. (shrink)

A working hypothesis of computationalism is that Mind arises, not from the intrinsic nature of the causal properties of particular forms of matter, but from the organization of matter. If this hypothesis is correct, then a wide range of physical systems (e.g. optical, chemical, various hybrids, etc.) should support Mind, especially computers, since they have the capability to create/manipulate organizations of bits of arbitrarily complexity and dynamics. In any particular computer, these bit patterns are quite physical, but their particular (...) physicality is considered irrelevant (since they could be replaced by other physical substrata). (shrink)

There is a gap between two different modes of computation: the symbolic mode and the subsymbolic (neuron-like) mode. The aim of this paper is to overcome this gap by viewing symbolism as a high-level description of the properties of (a class of) neural networks. Combining methods of algebraic semantics and non-monotonic logic, the possibility of integrating both modes of viewing cognition is demonstrated. The main results are (a) that certain activities of connectionist networks can be interpreted as (...) non-monotonic inferences, and (b) that there is a strict correspondence between the coding of knowledge in Hopfield networks and the knowledge representation in weight-annotated Poole systems. These results show the usefulness of non-monotonic logic as a descriptive and analytic tool for analyzing emerging properties of connectionist networks. Assuming an exponential development of the weight function, the present account relates to optimality theory – a general framework that aims to integrate insights from symbolism and connectionism. The paper concludes with some speculations about extending the present ideas. (shrink)

Although connectionism is advocated by its proponents as an alternative to the classical computational theory of mind, doubts persist about its _computational_ credentials. Our aim is to dispel these doubts by explaining how connectionist networks compute. We first develop a generic account of computation—no easy task, because computation, like almost every other foundational concept in cognitive science, has resisted canonical definition. We opt for a characterisation that does justice to the explanatory role of computation in cognitive science. Next we (...) examine what might be regarded as the “conventional” account of connectionist computation. We show why this account is inadequate and hence fosters the suspicion that connectionist networks aren’t genuinely computational. Lastly, we turn to the principal task of the paper: the development of a more robust portrait of connectionist computation. The basis of this portrait is an explanation of the representational capacities of connection weights, supported by an analysis of the weight configurations of a series of simulated neural networks. (shrink)

Intended to introduce readers to the use of artificial neural networks in the study of music, this volume contains numerous case studies and research findings that address problems related to identifying scales, keys, classifying musical chords, and learning jazz chord progressions. A detailed analysis of networks is provided for each case study which together demonstrate that focusing on the internal structure of trained networks could yield important contributions to the field of music cognition.

Choice Outstanding Academic Title, 1996. In hundreds of articles by experts from around the world, and in overviews and "road maps" prepared by the editor, The Handbook of Brain Theory and Neural Networkscharts the immense progress made in recent years in many specific areas related to two great questions: How does the brain work? and How can we build intelligent machines? While many books have appeared on limited aspects of one subfield or another of brain theory and neural (...) networks, the Handbookcovers the entire sweep of topics—from detailed models of single neurons, analyses of a wide variety of biological neural networks, and connectionist studies of psychology and language, to mathematical analyses of a variety of abstract neural networks, and technological applications of adaptive, artificial neural networks. The excitement, and the frustration, of these topics is that they span such a broad range of disciplines including mathematics, statistical physics and chemistry, neurology and neurobiology, and computer science and electrical engineering as well as cognitive psychology, artificial intelligence, and philosophy. Thus, much effort has gone into making the Handbookaccessible to readers with varied backgrounds while still providing a clear view of much of the recent, specialized research in specific topics. The heart of the book, part III, comprises of 267 original articles by leaders in the various fields, arranged alphabetically by title. Parts I and II, written by the editor, are designed to help readers orient themselves to this vast range of material. Part I, Background, introduces several basic neural models, explains how the present study of brain theory and neural networks integrates brain theory, artificial intelligence, and cognitive psychology, and provides a tutorial on the concepts essential for understanding neural networks as dynamic, adaptive systems. Part II, Road Maps, provides entry into the many articles of part III through an introductory "Meta-Map" and twenty-three road maps, each of which tours all the Part III articles on the chosen theme. (shrink)

Despite the lack of invariance problem (the many‐to‐many mapping between acoustics and percepts), human listeners experience phonetic constancy and typically perceive what a speaker intends. Most models of human speech recognition (HSR) have side‐stepped this problem, working with abstract, idealized inputs and deferring the challenge of working with real speech. In contrast, carefully engineered deep learning networks allow robust, real‐world automatic speech recognition (ASR). However, the complexities of deep learning architectures and training regimens make it difficult to use them (...) to provide direct insights into mechanisms that may support HSR. In this brief article, we report preliminary results from a two‐layer network that borrows one element from ASR, long short‐term memory nodes, which provide dynamic memory for a range of temporal spans. This allows the model to learn to map real speech from multiple talkers to semantic targets with high accuracy, with human‐like timecourse of lexical access and phonological competition. Internal representations emerge that resemble phonetically organized responses in human superior temporal gyrus, suggesting that the model develops a distributed phonological code despite no explicit training on phonetic or phonemic targets. The ability to work with real speech is a major advance for cognitive models of HSR. (shrink)

Face recognition is a computationally challenging classification task. Deep convolutional neural networks (DCNNs) are brain‐inspired algorithms that have recently reached human‐level performance in face and object recognition. However, it is not clear to what extent DCNNs generate a human‐like representation of face identity. We have recently revealed a subset of facial features that are used by humans for face recognition. This enables us now to ask whether DCNNs rely on the same facial information and whether this human‐like representation (...) depends on a system that is optimized for face identification. In the current study, we examined the representation of DCNNs of faces that differ in features that are critical or non‐critical for human face recognition. Our findings show that DCNNs optimized for face identification are tuned to the same facial features used by humans for face recognition. Sensitivity to these features was highly correlated with performance of the DCNN on a benchmark face recognition task. Moreover, sensitivity to these features and a view‐invariant face representation emerged at higher layers of a DCNN optimized for face recognition but not for object recognition. This finding parallels the division to a face and an object system in high‐level visual cortex. Taken together, these findings validate human perceptual models of face recognition, enable us to use DCNNs to test predictions about human face and object recognition as well as contribute to the interpretability of DCNNs. (shrink)

Legal text retrieval serves as a key component in a wide range of legal text processing tasks such as legal question answering, legal case entailment, and statute law retrieval. The performance of legal text retrieval depends, to a large extent, on the representation of text, both query and legal documents. Based on good representations, a legal text retrieval model can effectively match the query to its relevant documents. Because legal documents often contain long articles and only some parts are relevant (...) to queries, it is quite a challenge for existing models to represent such documents. In this paper, we study the use of attentive neural network-based text representation for statute law document retrieval. We propose a general approach using deep neural networks with attention mechanisms. Based on it, we develop two hierarchical architectures with sparse attention to represent long sentences and articles, and we name them Attentive CNN and Paraformer. The methods are evaluated on datasets of different sizes and characteristics in English, Japanese, and Vietnamese. Experimental results show that: (i) Attentive neural methods substantially outperform non-neural methods in terms of retrieval performance across datasets and languages; (ii) Pretrained transformer-based models achieve better accuracy on small datasets at the cost of high computational complexity while lighter weight Attentive CNN achieves better accuracy on large datasets; and (iii) Our proposed Paraformer outperforms state-of-the-art methods on COLIEE dataset, achieving the highest recall and F2 scores in the top-N retrieval task. (shrink)

In artificial intelligence, recent research has demonstrated the remarkable potential of Deep Convolutional Neural Networks (DCNNs), which seem to exceed state-of-the-art performance in new domains weekly, especially on the sorts of very difficult perceptual discrimination tasks that skeptics thought would remain beyond the reach of artificial intelligence. However, it has proven difficult to explain why DCNNs perform so well. In philosophy of mind, empiricists have long suggested that complex cognition is based on information derived from sensory experience, often (...) appealing to a faculty of abstraction. Rationalists have frequently complained, however, that empiricists never adequately explained how this faculty of abstraction actually works. In this paper, I tie these two questions together, to the mutual benefit of both disciplines. I argue that the architectural features that distinguish DCNNs from earlier neural networks allow them to implement a form of hierarchical processing that I call “transformational abstraction”. Transformational abstraction iteratively converts sensory-based representations of category exemplars into new formats that are increasingly tolerant to “nuisance variation” in input. Reflecting upon the way that DCNNs leverage a combination of linear and non-linear processing to efficiently accomplish this feat allows us to understand how the brain is capable of bi-directional travel between exemplars and abstractions, addressing longstanding problems in empiricist philosophy of mind. I end by considering the prospects for future research on DCNNs, arguing that rather than simply implementing 80s connectionism with more brute-force computation, transformational abstraction counts as a qualitatively distinct form of processing ripe with philosophical and psychological significance, because it is significantly better suited to depict the generic mechanism responsible for this important kind of psychological processing in the brain. (shrink)

“Rationality” in Simon's “bounded rationality” is the principle that humans make decisions on the basis of step-by-step reasoning using systematic rules of logic to maximize utility. “Bounded rationality” is the observation that the ability of a human brain to handle algorithmic complexity and large quantities of data is limited. Bounded rationality, in other words, treats a decision maker as a machine carrying out computations with limited resources. Under the principle of embodied cognition, a cognitive mind is an interactive machine. Turing-Church (...) computations are not interactive, and interactive machines can accomplish things that no Turing-Church computation can accomplish. Hence, if “rationality” is computation, and “bounded rationality” is computation with limited complexity, then “embodied bounded rationality” is both more limited than computation and more powerful. By embracing interaction, embodied bounded rationality can accomplish things that Turing-Church computation alone cannot. Deep neural networks, which have led to a revolution in artificial intelligence, are both interactive and not fundamentally algorithmic. Hence, their ability to mimic some cognitive capabilities far better than prior algorithmic techniques based on symbol manipulation provides empirical evidence for the principle of embodied bounded rationality. (shrink)

The book is thought-provoking and informative, wide in scope while also being technically detailed, and still relevant to modem AI [at least as of 1998, the time of writing this review, but probably also at the time of posting this entry here, 2023] even though it was published in 1989. This relevance lies mainly in the book’s advocacy of distributed computation at multiple levels of description, its combining of neural networks and other techniques, its emphasis on the interplay (...) between action and perception, and its particular approach to natural language processing. The criticisms I have occasionally made above are relatively minor, and one of the major shortcomings of schema theory at least at the time of the book’s publication -- namely the lack of a detailed, preferably neural, mechanism for schema instantiation -- is one of which Arbib is well aware. The book is well produced, for the most part clearly written, and has useful summaries at the beginning of each chapter. Both general readers and AI researchers (mainly but not exclusively those working on robotics, vision and neural networks) could benefit from reading the book. (shrink)

Decades of psychological research have been aimed at modeling how people learn features and categories. The empirical validation of these theories is often based on artificial stimuli with simple representations. Recently, deep neural networks have reached or surpassed human accuracy on tasks such as identifying objects in natural images. These networks learn representations of real‐world stimuli that can potentially be leveraged to capture psychological representations. We find that state‐of‐the‐art object classification networks provide surprisingly accurate predictions of (...) human similarity judgments for natural images, but they fail to capture some of the structure represented by people. We show that a simple transformation that corrects these discrepancies can be obtained through convex optimization. We use the resulting representations to predict the difficulty of learning novel categories of natural images. Our results extend the scope of psychological experiments and computational modeling by enabling tractable use of large natural stimulus sets. (shrink)

The importance of the Stability Problem in neurocomputing is discussed, as well as the need for the study of infinite networks. Stability must be the key ingredient in the solution of a problem by a neural network without external intervention. Infinite discrete networks seem to be the proper objects of study for a theory of neural computability which aims at characterizing problems solvable, in principle, by a neural network. Precise definitions of such problems and their (...) solutions are given. Some consequences are explored, in particular, the neural unsolvability of the Stability Problem for neural networks. (shrink)