Tetris provides a difficult, dynamic task environment within which some people are novices and others, after years of work and practice, become extreme experts. Here we study two core skills; namely, choosing the goal or objective function that will maximize performance and a feature-based analysis of the current game board to determine where to place the currently falling zoid so as to maximize the goal. In Study 1, we build cross-entropy reinforcement learning models to determine whether different goals (...) result in different feature weights. Two of these optimization strategies quickly rise to performance plateaus, whereas two others continue toward higher but more jagged heights. In Study 2, we compare the zoid placement decisions made by our best CERL models with those made by 67 human players. Across 370,131 human game episodes, two CERL models picked the same zoid placements as our lowest scoring human for 43% of the placements and as our three best scoring experts for 65% of the placements. Our findings suggest that people focus on maximizing points, not number of lines cleared or number of levels reached. They also show that goal choice influences the choice of zoid placements for CERLs and suggest that the same is true of humans. Tetris has a repetitive task structure that makes Tetris more tractable and more like a traditional experimental psychology paradigm than many more complex games or tasks. Hence, although complex, Tetris is not overwhelmingly complex and presents a right-sized challenge to cognitive theories, especially those of integrated cognitive systems. (shrink)

Tetris provides a difficult, dynamic task environment within which some people are novices and others, after years of work and practice, become extreme experts. Here we study two core skills; namely, choosing the goal or objective function that will maximize performance and a feature-based analysis of the current game board to determine where to place the currently falling zoid so as to maximize the goal. In Study 1, we build cross-entropy reinforcement learning models to determine whether different goals (...) result in different feature weights. Two of these optimization strategies quickly rise to performance plateaus, whereas two others continue toward higher but more jagged heights. In Study 2, we compare the zoid placement decisions made by our best CERL models with those made by 67 human players. Across 370,131 human game episodes, two CERL models picked the same zoid placements as our lowest scoring human for 43% of the placements and as our three best scoring experts for 65% of the placements. Our findings suggest that people focus on maximizing points, not number of lines cleared or number of levels reached. They also show that goal choice influences the choice of zoid placements for CERLs and suggest that the same is true of humans. Tetris has a repetitive task structure that makes Tetris more tractable and more like a traditional experimental psychology paradigm than many more complex games or tasks. Hence, although complex, Tetris is not overwhelmingly complex and presents a right-sized challenge to cognitive theories, especially those of integrated cognitive systems. (shrink)

Relational database management system is the most popular database system. It is important to maintain data security from information leakage and data corruption. RDBMS can be attacked by an outsider or an insider. It is difficult to detect an insider attack because its patterns are constantly changing and evolving. In this paper, we propose an adaptive database intrusion detection system that can be resistant to potential insider misuse using evolutionary reinforcement learning, which combines reinforcement learning and (...) evolutionary learning. The model consists of two neural networks, an evaluation network and an action network. The action network detects the intrusion, and the evaluation network provides feedback to the detection of the action network. Evolutionary learning is effective for dynamic patterns and atypical patterns, and reinforcement learning enables online learning. Experimental results show that the performance for detecting abnormal queries improves as the proposed model learns the intrusion adaptively using Transaction Processing performance Council-E scenario-based virtual query data. The proposed method achieves the highest performance at 94.86%, and we demonstrate the usefulness of the proposed method by performing 5-fold cross-validation. (shrink)

Rapid and precise air operation mission planning is a key technology in unmanned aerial vehicles autonomous combat in battles. In this paper, an end-to-end UAV intelligent mission planning method based on deep reinforcement learning is proposed to solve the shortcomings of the traditional intelligent optimization algorithm, such as relying on simple, static, low-dimensional scenarios, and poor scalability. Specifically, the suppression of enemy air defense mission planning is described as a sequential decision-making problem and formalized as a Markov decision (...) process. Then, the SEAD intelligent planning model based on the proximal policy optimization algorithm is established and a general intelligent planning architecture is proposed. Furthermore, three policy training tricks, i.e., domain randomization, maximizing policy entropy, and underlying network parameter sharing, are introduced to improve the learning performance and generalizability of PPO. Experiments results show that the model in this work is efficient and stable, and can be adapted to the unknown continuous high-dimensional environment. It can be concluded that the UAV intelligent mission planning model based on DRL has powerful intelligent planning performance, and provides a new idea for researching UAV autonomy. (shrink)

We evaluate here the performance of four models of cross-situational word learning: two global models, which extract and retain multiple referential alternatives from each word occurrence; and two local models, which extract just a single referent from each occurrence. One of these local models, dubbed Pursuit, uses an associative learning mechanism to estimate word-referent probability but pursues and tests the best referent-meaning at any given time. Pursuit is found to perform as well as global models under many conditions (...) extracted from naturalistic corpora of parent-child interactions, even though the model maintains far less information than global models. Moreover, Pursuit is found to best capture human experimental findings from several relevant cross-situational word-learning experiments, including those of Yu and Smith (), the paradigm example of a finding believed to support fully global cross-situational models. Implications and limitations of these results are discussed, most notably that the model characterizes only the earliest stages of word learning, when reliance on the co-occurring referent world is at its greatest. (shrink)

The detection and categorization of animate motions is a crucial task underlying social interaction and perceptual decision making. Neural representations of perceived animate objects are partially located in the primate cortical region STS, which is a region that receives convergent input from intermediate-level form and motion representations. Populations of STS cells exist which are selectively responsive to specific animated motion sequences, such as walkers. It is still unclear how and to what extent form and motion information contribute to the generation (...) of such representations and what kind of mechanisms are involved in the learning processes. The article develops a cortical model architecture for the unsupervised learning of animated motion sequence representations. We demonstrate how the model automatically selects significant motion patterns as well as meaningful static form prototypes characterized by a high degree of articulation. Such key poses are selectively reinforced during learning through a cross talk between the motion and form processing streams. Furthermore, we show how sequence-selective representations are learned in STS by fusing static form and motion input from the segregated bottom-up driving input streams. Cells in STS, in turn, feed their activities recurrently to their input sites along top-down signal pathways. We show how such learned feedback connections enable predictions about future input as anticipation generated by sequence-selective STS cells. Network simulations demonstrate the computational capacity of the proposed model by reproducing several experimental findings from neurosciences and by accounting for recent behavioral data. (shrink)

For a robot to be capable of development it must be able to explore its environment and learn from its experiences. It must find opportunities to experience the unfamiliar in ways that reveal properties valid beyond the immediate context. In this paper, we develop a novel method for using the rhythm of everyday actions as a basis for identifying the characteristic appearance and sounds associated with objects, people, and the robot itself. Our approach is to identify and segment groups of (...) signals in individual modalities based on their rhythmic variation, then to identify and bind causally-related groups of signals across different modalities. By including proprioception as a modality, this cross-modal binding method applies to the robot itself, and we report a series of experiments in which the robot learns about the characteristics of its own body. (shrink)

The COVID-19 pandemic rapidly led to changes in the mode of teaching, learning and assessments in most tertiary institutions worldwide. Notably, non-invigilated summative assessments became predominant. These changes heightened anxiety and depression, especially among individuals with less resilient coping mechanism. We explored the perceptions and experiences of mental health difficulties of students in tertiary education regarding non-invigilated alternative assessments in comparison to invigilated assessments. A pragmatic, mixed method cross sectional design was conducted online via Qualtrics. Thematic analysis of text (...) was carried out using NVivo 12. In the quantitative analysis, univariable and multivariable ordinal logistic models were used to examine the potential factors for preference among students in higher education. A total of 380 Nursing and Social Science students responded to the survey. Approximately 77% of students perceived non-invigilated assessments to be less stressful compared to invigilated exams. Age, course of study, stage of studies, and number of units enrolled per semester were identified as significant drivers for students’ perceived preference for non-invigilated assessments. There was an inverse relationship between the perception of stress associated with invigilated exams and the age of students. For instance, students aged between 18-24 were 5 times more likely to prefer non-invigilated exams compared to those aged 55 or more. Comparatively, students in early stages of studies had higher preference for non-invigilated assessments. However, there was a preference reversal for students enrolled in 2 or less units per semester. Social sciences students were two times more likely to prefer non-invigilated examinations to invigilated examinations compared to nursing students. The findings reinforce the use of alternative assessments in higher education as a mitigating agency to lessen the mental health burden of tertiary students in post COVID-19 era. (shrink)

Ship-type identification in a maritime context can be critical to the authorities to control the activities being carried out. Although Automatic Identification Systems has been mandatory for certain vessels, if a vessel does not have them voluntarily or not, it can lead to a whole set of problems, which is why the use of tracking alternatives such as radar is fully complementary for a vessel monitoring systems. However, radars provide positions, but not what they are detecting. Having systems capable of (...) adding categorical information to radar detections of vessels makes it possible to increase control of the activities being carried out, improve safety in maritime traffic, and optimize on-site inspection resources on the part of the authorities. This paper addresses the binary classification problem (fishing ships versus all other vessels) using unbalanced data from real vessel trajectories. It is performed from a deep learning approach comparing two of the main trends, Convolutional Neural Networks and Long Short-Term Memory. In this paper, it is proposed the weighted cross-entropy methodology and compared with classical data balancing strategies. Both networks show high performance when applying weighted cross-entropy compared with the classical machine learning approaches and classical balancing techniques. This work is shown to be a novel approach to the international problem of identifying fishing ships without context. (shrink)

Proclaimed the “greenest television programme in the world,” the award-winning soap opera Coronation Street is seen as an industry success story. This paper explores how the integration of carbon literacy training led to a widespread transformational change of practice within Coronation Street. Using the theoretical lens of Communities of Practice, this study examines the nature of social learning and the enablers and barriers to change within the organization. Specifically, how boundary spanning practices, objects and people led to the transformation (...) on both a personal and group level. Based on a qualitative analysis of 22 interviews with Heads of Departments and other staff, the paper argues that CLT is a boundary practice which has evolved into a boundary spanning CoP. The importance of infrastructures supporting boundary objects and practices is highlighted as reinforcers of the CLT, both as a boundary object and a community, with the “ultimate” boundary spanning object being the show. A significant enabler in social learning and change in practice is the creation of discursive and creative space, both within CoP and across the boundaries. Findings also highlight the role of “self” in the process of social learning and organizational change. Distinct patterns emerged in the relationship between self-identity, social learning and change across a range of boundary objects, practices and communities both in the CLT and CoP. This suggests that in a diverse social learning setting such as CLT there are different transformational catalysts within the CoP and these identities can influence how knowledge is translated into practice. (shrink)

Cross entropy measure is one of the best way to calculate the divergence of any variable from the priori one variable. We define a new cross entropy measure under interval neutrosophic set environment.

Animals routinely adapt to changes in the environment in order to survive. Though reinforcement learning may play a role in such adaptation, it is not clear that it is the only mechanism involved, as it is not well suited to producing rapid, relatively immediate changes in strategies in response to environmental changes. This research proposes that counterfactual reasoning might be an additional mechanism that facilitates change detection. An experiment is conducted in which a task state changes over time (...) and the participants had to detect the changes in order to perform well and gain monetary rewards. A cognitive model is constructed that incorporates reinforcement learning with counterfactual reasoning to help quickly adjust the utility of task strategies in response to changes. The results show that the model can accurately explain human data and that counterfactual reasoning is key to reproducing the various effects observed in this change detection paradigm. (shrink)

This fifth volume on Advances and Applications of DSmT for Information Fusion collects theoretical and applied contributions of researchers working in different fields of applications and in mathematics, and is available in open-access. The collected contributions of this volume have either been published or presented after disseminating the fourth volume in 2015 in international conferences, seminars, workshops and journals, or they are new. The contributions of each part of this volume are chronologically ordered. First Part of this book presents some (...) theoretical advances on DSmT, dealing mainly with modified Proportional Conflict Redistribution Rules (PCR) of combination with degree of intersection, coarsening techniques, interval calculus for PCR thanks to set inversion via interval analysis (SIVIA), rough set classifiers, canonical decomposition of dichotomous belief functions, fast PCR fusion, fast inter-criteria analysis with PCR, and improved PCR5 and PCR6 rules preserving the (quasi-)neutrality of (quasi-)vacuous belief assignment in the fusion of sources of evidence with their Matlab codes. Because more applications of DSmT have emerged in the past years since the apparition of the fourth book of DSmT in 2015, the second part of this volume is about selected applications of DSmT mainly in building change detection, object recognition, quality of data association in tracking, perception in robotics, risk assessment for torrent protection and multi-criteria decision-making, multi-modal image fusion, coarsening techniques, recommender system, levee characterization and assessment, human heading perception, trust assessment, robotics, biometrics, failure detection, GPS systems, inter-criteria analysis, group decision, human activity recognition, storm prediction, data association for autonomous vehicles, identification of maritime vessels, fusion of support vector machines (SVM), Silx-Furtif RUST code library for information fusion including PCR rules, and network for ship classification. Finally, the third part presents interesting contributions related to belief functions in general published or presented along the years since 2015. These contributions are related with decision-making under uncertainty, belief approximations, probability transformations, new distances between belief functions, non-classical multi-criteria decision-making problems with belief functions, generalization of Bayes theorem, image processing, data association, entropy and cross-entropy measures, fuzzy evidence numbers, negator of belief mass, human activity recognition, information fusion for breast cancer therapy, imbalanced data classification, and hybrid techniques mixing deep learning with belief functions as well. (shrink)

There is an apparent connection between reinforcement learning and agency. Artificial entities controlled by reinforcement learning algorithms are standardly referred to as agents, and the mainstream view in the psychology and neuroscience of agency is that humans and other animals are reinforcement learners. This article examines this connection, focusing on artificial reinforcement learning systems and assuming that there are various forms of agency. Artificial reinforcement learning systems satisfy plausible conditions for minimal (...) agency, and those which use models of the environment to perform forward search are capable of a form of agency which may reasonably be called action for reasons. (shrink)

Research-based but highly accessible, this fresh, contemporary, and engaging volume helps students appreciate the science of psychology and understand how its principles apply to their own lives. Key features: Contemporary perspectives and references: giving careful consideration to the field's historical foundations, Psychology: Contemporary Perspectives provides a unique balance of traditional and contemporary perspectives. This approach invites students to develop a modern appraisal of psychology. Current research: the book covers the latest in evolutionary psychology and behavior genetics, ecological and evolutionary theories (...) of learning, cross-cultural work in cognition, the latest neuroscience data, and endophenotype research in the genetic causes of schizophrenia. Clear and compelling writing: exceptionally well written, Psychology: Contemporary Perspectives uses stories to help students connect with the principles of psychology. Rich pedagogy: all running features are integrated into the main body of the text, helping to maintain the flow of the narrative--and the attention of students! "Freeze Frame" snapshots underscoring the theme of each chapter tell the compelling stories of real-life individuals. "Living Psychology" applies psychology to students' everyday lives, helping them understand the benefits of what they are studying. "Critical Thinking about Psychology" sections give students the tools they need in order to effectively and objectively interpret research. "At the Forefront" highlights new and influential research in the field. Carefully selected study tools: placing an emphasis on "review, retrieve, and learn", Psychology: Contemporary Perspectives takes a "back-to-basics" approach to study tools. "In Summary" sections provide clear, concise chapter descriptions, offering students powerful tools for review. "Retrieve!" questions encourage students to test their knowledge of what they have just read. Each question includes the page numbers on which the relevant material was first presented, aiding in additional review and reinforcing learning. (shrink)

We provide a short review on the recent and near-future developments of computational processing of emotion in the voice, highlighting self-learning of representations moving continuously away from traditional expert-crafted or brute-forced feature representations to end-to-end learning, a movement towards the coupling of analysis and synthesis of emotional voices to foster better mutual understanding, weakly supervised learning at a large scale, transfer learning from related domains such as speech recognition or cross-modal transfer learning, and reinforced (...) class='Hi'>learning through interactive applications at a large scale. For each of these trends, we shortly explain their implications and potential use such as for interpretation in psychological studies and usage in digital health and digital psychology applications. We also discuss further potential development. (shrink)

We motivate the desire for self-driving and explain its potential and limitations, and explore the need for—and potential implementation of—morals, ethics, and other value systems as complementary “capabilities” to the Deep Technologies behind self-driving. We consider how the incorporation of such systems may drive or slow adoption of high automation within vehicles. First, we explore the role for morals, ethics, and other value systems in self-driving through a representative hypothetical dilemma faced by a self-driving car. Through the lens of engineering, (...) we explain in simple terms common moral and ethical frameworks including utilitarianism, deontology, and virtue ethics before characterizing their relationship to the fundamental algorithms enabling self-driving. The concepts of behavior cloning, state-based modeling, and reinforcement learning are introduced, with some algorithms being more suitable for the implementation of value systems than others. We touch upon the contemporary cross-disciplinary landscape of morals and ethics in self-driving systems from a joint philosophical and technical perspective, and close with considerations for practitioners and the public, particularly as individuals may not appreciate the nuance and complexity of using imperfect information to navigate diverse scenarios and tough-to-quantify value systems, while “typical” software development reduces complex problems to black and white decision-making. (shrink)

Yang returns to the problem of referential ambiguity, addressed in the opening paper by Gleitman and Trueswell. Using a computational approach, he argues that “big data” approaches to resolving referential ambiguity are destined to fail, because of the inevitable computational explosion needed to keep track of contextual associations present when a word is uttered. Yang tests several computational models, two of which depend on one‐trial learning, as described in Gleitman and Trueswell’s paper. He concludes that such models outperform cross‐situational (...) learning models, thereby formalizing, reinforcing, and advancing the conclusions offered by Gleitman and Trueswell. (shrink)

A deep reinforcement learning-based computational guidance method is presented, which is used to identify and resolve the problem of collision avoidance for a variable number of fixed-wing UAVs in limited airspace. The cooperative guidance process is first analyzed for multiple aircraft by formulating flight scenarios using multiagent Markov game theory and solving it by machine learning algorithm. Furthermore, a self-learning framework is established by using the actor-critic model, which is proposed to train collision avoidance decision-making neural (...) networks. To achieve higher scalability, the neural network is customized to incorporate long short-term memory networks, and a coordination strategy is given. Additionally, a simulator suitable for multiagent high-density route scene is designed for validation, in which all UAVs run the proposed algorithm onboard. Simulated experiment results from several case studies show that the real-time guidance algorithm can reduce the collision probability of multiple UAVs in flight effectively even with a large number of aircraft. (shrink)

We investigate how Reinforcement Learning methods can be used to solve the production selection and production ordering problem in ACT‐R. We focus on four algorithms from the Q learning family, tabular Q and three versions of Deep Q Networks, as well as the ACT‐R utility learning algorithm, which provides a baseline for the Q algorithms. We compare the performance of these five algorithms in a range of lexical decision tasks framed as sequential decision problems.

Curiosity and creativity are expressions of the trade-off between leveraging that with which we are familiar or seeking out novelty. Through the computational lens of reinforcement learning, we describe how formulating the value of information seeking and generation via their complementary effects on planning horizons formally captures a range of solutions to striking this balance.

In this opinionated review, I draw attention to some of the contributions reinforcement learning can make to questions in the philosophy of mind. In particular, I highlight reinforcement learning's foundational emphasis on the role of reward in agent learning, and canvass two ways in which the framework may advance our understanding of perception and motivation.

I argue for the role of reinforcement learning in the philosophy of mind. To start, I make several assumptions about the nature of reinforcement learning and its instantiation in minds like ours. I then review some of the contributions of reinforcement learning methods have made across the so-called 'decision sciences.' Finally, I show how principles from reinforcement learning can shape philosophical debates regarding the nature of perception and characterisations of desire.

A fundamental question in reading research concerns whether attention is allocated strictly serially, supporting lexical processing of one word at a time, or in parallel, supporting concurrent lexical processing of two or more words (Reichle, Liversedge, Pollatsek, & Rayner, 2009). The origins of this debate are reviewed. We then report three simulations to address this question using artificial reading agents (Liu & Reichle, 2010; Reichle & Laurent, 2006) that learn to dynamically allocate attention to 1–4 words to “read” as efficiently (...) as possible. These simulation results indicate that the agents strongly preferred serial word processing, although they occasionally attended to more than one word concurrently. The reason for this preference is discussed, along with implications for the debate about how humans allocate attention during reading. (shrink)

Sequential action makes up the bulk of human daily activity, and yet much remains unknown about how people learn such actions. In one motor learning paradigm, the serial reaction time (SRT) task, people are taught a consistent sequence of button presses by cueing them with the next target response. However, the SRT task only records keypress response times to a cued target, and thus it cannot reveal the full time‐course of motion, including predictive movements. This paper describes a mouse (...) movement trajectory SRT task in which the cursor must be moved to a cued location. We replicated keypress SRT results, but also found that predictive movement—before the next cue appears—increased during the experiment. Moreover, trajectory analyses revealed that people developed a centering strategy under uncertainty. In a second experiment, we made prediction explicit, no longer cueing targets. Thus, participants had to explore the response alternatives and learn via reinforcement, receiving rewards and penalties for correct and incorrect actions, respectively. Participants were not told whether the sequence of stimuli was deterministic, nor if it would repeat, nor how long it was. Given the difficulty of the task, it is unsurprising that some learners performed poorly. However, many learners performed remarkably well, and some acquired the full 10‐item sequence within 10 repetitions. Comparing the high‐ and low‐performers’ detailed results in this reinforcement learning (RL) task with the first experiment's cued trajectory SRT task, we found similarities between the two tasks, suggesting that the effects in Experiment 1 are due to predictive, rather than reactive processes. Finally, we found that two standard model‐free reinforcement learning models fit the high‐performing participants, while the four low‐performing participants provide better fit with a simple negative recency bias model. (shrink)

Sequential action makes up the bulk of human daily activity, and yet much remains unknown about how people learn such actions. In one motor learning paradigm, the serial reaction time (SRT) task, people are taught a consistent sequence of button presses by cueing them with the next target response. However, the SRT task only records keypress response times to a cued target, and thus it cannot reveal the full time‐course of motion, including predictive movements. This paper describes a mouse (...) movement trajectory SRT task in which the cursor must be moved to a cued location. We replicated keypress SRT results, but also found that predictive movement—before the next cue appears—increased during the experiment. Moreover, trajectory analyses revealed that people developed a centering strategy under uncertainty. In a second experiment, we made prediction explicit, no longer cueing targets. Thus, participants had to explore the response alternatives and learn via reinforcement, receiving rewards and penalties for correct and incorrect actions, respectively. Participants were not told whether the sequence of stimuli was deterministic, nor if it would repeat, nor how long it was. Given the difficulty of the task, it is unsurprising that some learners performed poorly. However, many learners performed remarkably well, and some acquired the full 10‐item sequence within 10 repetitions. Comparing the high‐ and low‐performers’ detailed results in this reinforcement learning (RL) task with the first experiment's cued trajectory SRT task, we found similarities between the two tasks, suggesting that the effects in Experiment 1 are due to predictive, rather than reactive processes. Finally, we found that two standard model‐free reinforcement learning models fit the high‐performing participants, while the four low‐performing participants provide better fit with a simple negative recency bias model. (shrink)

This paper presents a GA-based multi-agent reinforce- ment learning bidding approach (GMARLB) for perform- ing multi-agent reinforcement learning. GMARLB inte- grates reinforcement learning, bidding and genetic algo- rithms. The general idea of our multi-agent systems is as follows: There are a number of individual agents in a team, each agent of the team has two modules: Q module and CQ module. Each agent can select actions to be performed at each step, which are done by (...) the Q module. While the CQ module determines at each step whether the agent should continue or relinquish control. Once an agent relinquishes its control, a new agent is selected by bidding algorithms. We applied GA-based GMARLB to the Backgammon game. The experimental results show GMARLB can achieve a su- perior level of performance in game-playing, outperforming PubEval, while the system uses zero built-in knowledge. (shrink)

Cross-situational word learning, like any statistical learning problem, involves tracking the regularities in the environment. However, the information that learners pick up from these regularities is dependent on their learning mechanism. This article investigates the role of one type of mechanism in statistical word learning: competition. Competitive mechanisms would allow learners to find the signal in noisy input and would help to explain the speed with which learners succeed in statistical learning tasks. Because cross-situational word (...) learning provides information at multiple scales—both within and across trials/situations—learners could implement competition at either or both of these scales. A series of four experiments demonstrate that cross-situational learning involves competition at both levels of scale, and that these mechanisms interact to support rapid learning. The impact of both of these mechanisms is considered from the perspective of a process-level understanding of cross-situational learning. (shrink)

In reinforcement learning, a decision maker searching for the most rewarding option is often faced with the question: What is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: How can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and we describe an equivalence between the Bayesian (...) model and temporal difference learning algorithms that have been proposed as models of RL in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional RL algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty. (shrink)

The area of smart power grids needs to constantly improve its efficiency and resilience, to provide high quality electrical power in a resilient grid, while managing faults and avoiding failures. Achieving this requires high component reliability, adequate maintenance, and a studied failure occurrence. Correct system operation involves those activities and novel methodologies to detect, classify, and isolate faults and failures and model and simulate processes with predictive algorithms and analytics. In this paper, we showcase the application of a complex-adaptive, self-organizing (...) modeling method, and Probabilistic Boolean Networks, as a way towards the understanding of the dynamics of smart grid devices, and to model and characterize their behavior. This work demonstrates that PBNs are equivalent to the standard Reinforcement Learning Cycle, in which the agent/model has an interaction with its environment and receives feedback from it in the form of a reward signal. Different reward structures were created to characterize preferred behavior. This information can be used to guide the PBN to avoid fault conditions and failures. (shrink)

Autonomous underwater vehicles are widely used to accomplish various missions in the complex marine environment; the design of a control system for AUVs is particularly difficult due to the high nonlinearity, variations in hydrodynamic coefficients, and external force from ocean currents. In this paper, we propose a controller based on deep reinforcement learning in a simulation environment for studying the control performance of the vectored thruster AUV. RL is an important method of artificial intelligence that can learn behavior (...) through trial-and-error interactions with the environment, so it does not need to provide an accurate AUV control model that is very hard to establish. The proposed RL algorithm only uses the information that can be measured by sensors inside the AUVs as the input parameters, and the outputs of the designed controller are the continuous control actions, which are the commands that are set to the vectored thruster. Moreover, a reward function is developed for deep RL controller considering different factors which actually affect the control accuracy of AUV navigation control. To confirm the algorithm’s effectiveness, a series of simulations are carried out in the designed simulation environment, which is a method to save time and improve efficiency. Simulation results prove the feasibility of the deep RL algorithm applied to the control system for AUV. Furthermore, our work also provides an optional method for robot control problems to deal with improving technology requirements and complicated application environments. (shrink)

We investigate a simple stochastic model of social network formation by the process of reinforcement learning with discounting of the past. In the limit, for any value of the discounting parameter, small, stable cliques are formed. However, the time it takes to reach the limiting state in which cliques have formed is very sensitive to the discounting parameter. Depending on this value, the limiting result may or may not be a good predictor for realistic observation times.

This paper addresses weighting and partitioning in complex reinforcement learning tasks, with the aim of facilitating learning. The paper presents some ideas regarding weighting of multiple agents and extends them into partitioning an input/state space into multiple regions with di erential weighting in these regions, to exploit di erential characteristics of regions and di erential characteristics of agents to reduce the learning complexity of agents (and their function approximators) and thus to facilitate the learning overall. (...) It analyzes, in reinforcement learning tasks, di erent ways of partitioning a task and using agents selectively based on partitioning. Based on the analysis, some heuristic methods are described and experimentally tested. We nd that some o -line heuristic methods performed the best, signi cantly better than single-agent models. (shrink)

We welcome Soltis' use of evolutionary signaling theory, but question his interpretations of colic as a signal of vigor and his explanation of abnormal high-pitched crying as a signal of poor infant quality. Instead, we suggest that these phenomena may be suboptimal by-products of a generally adaptive learning process by which infants adjust their crying levels in relation to parental responsiveness.

The brain-computer interface based on functional near-infrared spectroscopy has received more and more attention due to its vast application potential in emotion recognition. However, the relatively insufficient investigation of the feature extraction algorithms limits its use in practice. In this article, to improve the performance of fNIRS-based BCI, we proposed a method named R-CSP-E, which introduces EEG signals when computing fNIRS signals’ features based on transfer learning and ensemble learning theory. In detail, we used the Independent Component Analysis (...) algorithm for the correspondence between the sources of the two signals. We then introduced the EEG signals when computing the spatial filter based on a modified Common Spatial Pattern algorithm. Experimental results on public datasets show that the proposed method in this paper outperforms traditional methods without transfer. In general, the mean classification accuracy can be increased by up to 5%. To our knowledge, it is an innovation that we tried to apply transfer learning between EEG and fNIRS. Our study’s findings not only prove the potential of the transfer learning algorithm in cross-model brain-computer interface, but also offer a new and innovative perspective to research the hybrid brain-computer interface. (shrink)