The experience of a reward appears to enhance memory for recent prior events, adaptively making that information more available to guide future decision-making. Here, we tested whether reward enhances memory for associative item-location information and also whether the effect of reward spreads to other categorically-related but unrewarded items. Participants earned either points (Experiment 1) or money (Experiment 2) through a time-estimation reward task, during which stimuli-location pairings around a 2D-ring were shown followed by either high-value or low-value rewards. All (...) stimuli were then tested for location memory or recognition (yes/no), immediately and after a 24-hour delay. Across both experiments (combined analysis), there was a robust improvement in location memory following high-value rewards, even though evidence supporting this effect was reliable in Experiment 2 but not in Experiment 1. The memory-enhancing effect of reward was observed on both the immediate and delayed location-memory tests. Reward-enhanced memory for both directly rewarded stimuli and categorically related stimuli that were not directly rewarded. No reliable effect of reward value on yes/no recognition-memory performance was observed in either experiment. We hypothesise that reward enhances the consolidation of recent experience and conceptually related memories to make these more available for future decisions. (shrink)

1. Introduction 2. Reward-Guided Decision Making 3. Content in the Model 4. How to Deflate a Metarepresentational Reading Proust and Carruthers on metacognitive feelings 5. A Deflationary Treatment of RPEs? 5.1 Dispensing with prediction errors 5.2 What is use of the RPE focused on? 5.3 Alternative explanations—worldly correlates 5.4 Contrast cases 6. Conclusion Appendix: Temporal Difference Learning Algorithms.

Can an agent's intelligence level be negative? We extend the Legg-Hutter agent-environment framework to include punishments and argue for an affirmative answer to that question. We show that if the background encodings and Universal Turing Machine (UTM) admit certain Kolmogorov complexity symmetries, then the resulting Legg-Hutter intelligence measure is symmetric about the origin. In particular, this implies reward-ignoring agents have Legg-Hutter intelligence 0 according to such UTMs.

Unlike other creatures, humans developed the ability to cooperate with genetically unrelated strangers and a tendency to comply with social norms. However, humans deviate from social norms in various situations. This study used the modified ultimatum game to explore why humans deviate from social norms and how their prosocial behavior can be promoted. In Study 1, participants were asked to imagine working with an anonymous counterpart to complete a task and obtain a certain amount of money (e.g., ¥10). The computer (...) divided the money randomly in favor of the participant (e.g., 9:1 or 8:2). Participants should decide whether to accept or reject such a self-benefiting division. In the non-risk condition, an absolutely fair redivision of money would take place if participants reject self-benefiting division (e.g. 5:5 or 6:4). By contrast, in the risk condition, other-benefiting redivision of money (e.g., 1:9 or 2:8) would take place if participants rejected the self-benefiting division. Results involving 40 college students showed the main effect of condition. The frequency of accepting self-benefiting division in the risk condition was higher than that in the non-risk condition. As such, compliance with social norms is based on the preservation of material resources. In Study 2, we used economic or moral rewards to compensate for economic loss following compliance with the norm. Results involving 28 college students revealed a significant effect of compensation. The rewards, including moral praise, effectively decreased selfish choices. These findings extend previous studies on social norm compliance by emphasizing the importance of internal, fairness-based balance between material and moral needs, as well as the role of moral praise in promoting prosocial behavior. (shrink)

This qualitative study of 27 women animal activists examines the risks and rewards that accompany a commitment to animal rights activism. One of the common beliefs about animal rights activists is that their political choices are fanatic and unyielding, resulting in rigid self-denial. Contrary to this notion, the women in this study experienced both the pain and the joy of their transformation toward animal activism. Activism took an enormous toll on their personal relationships, careers, and emotional well being. They (...) struggled as friendships ended and family relationships suffered; some experienced harassment and abuse as a result of their efforts. Yet the women were just as likely to extol the rewards and pleasure gained from their participation in the cause of animal liberation. These included a heightened awareness of political issues, greater self-confidence, the feeling that they were making a difference in the world, and the joy of living a “more meaningful life.”. (shrink)

When faced with intertemporal choices, which have consequences that unfold over time, we often discount the future, preferring smaller immediate rewards often at the expense of long-term benefits. How psychologically connected one feels to one’s future self-influences such temporal discounting. Psychological connectedness consists in sharing psychological properties with past or future selves, but connectedness comes in degrees. If one feels that one is not psychologically connected to one’s future self, one views that self like a different person and is (...) less likely to wait for the future reward. Increasing perceived psychological connectedness to one’s future self may lead to more far-sighted decisions. Episodic prospection may help in this regard. Episodic prospection is our ability to ‘pre-experience’ the future by mentally simulating it, drawing on information from episodic memory and other sources. Episodic memory and prospection are thought to involve a special form of consciousness, which underpins the capacity to appreciate the connection between one’s past, present, and future selves. Simulating the future self through prospection may increase felt psychological connectedness and support future-oriented decision-making. Yet this is where a puzzle arises. The imagery of episodic memory and prospection is perspectival: often one views the visualised scenario from a detached perspective, seeing oneself from-the-outside as if viewing another person. The aim of this paper is to characterise how the perspectival imagery of prospection relates to psychological connectedness, and to show that even though such imagery involves a detached perspective it can still be used to help reward one’s future self. (shrink)

Systems of logico-probabilistic reasoning characterize inference from conditional assertions that express high conditional probabilities. In this paper we investigate four prominent LP systems, the systems _O, P_, _Z_, and _QC_. These systems differ in the number of inferences they licence _. LP systems that license more inferences enjoy the possible reward of deriving more true and informative conclusions, but with this possible reward comes the risk of drawing more false or uninformative conclusions. In the first part of the paper, we (...) present the four systems and extend each of them by theorems that allow one to compute almost-tight lower-probability-bounds for the conclusion of an inference, given lower-probability-bounds for its premises. In the second part of the paper, we investigate by means of computer simulations which of the four systems provides the best balance of reward versus risk. Our results suggest that system _Z_ offers the best balance. (shrink)

Rewards are reinforcement mechanisms that organizations use to shape desirable employee behaviors. However, rewards may also have unintended consequences, such as building expectations for receiving extra benefits and weakening employee barriers to unethical acts. This article investigates the dark side of the reward–behavior association, and exploring what is referred to as the reward–theft parity effect (RTPE). The authors hypothesize that receiving rewards induces a corresponding type of theft. In Study 1, survey results (n = 634) show initial (...) support for the RTPE between rewards and same‐category theft, and that perceptions of procedural justice attenuate this association. Study 2 involved a 2 (reward) × 2 (theft opportunity) between‐subjects experimental design (n = 300) and provides further support for the RTPE by revealing greater theft under a reward–theft parity condition than under a reward–theft disparity condition. Study 3 replicates these findings in a field experiment (n = 285) and finds that the RTPE is exacerbated when the reward–behavior task is perceived as unjust. Collectively, these findings provide new theoretical insights while also giving managers a more complete understanding of the dark side of behavioral reinforcement mechanisms. (shrink)

Can humans get arbitrarily capable reinforcement learning agents to do their bidding? Or will sufficiently capable RL agents always find ways to bypass their intended objectives by shortcutting their reward signal? This question impacts how far RL can be scaled, and whether alternative paradigms must be developed in order to build safe artificial general intelligence. In this paper, we study when an RL agent has an instrumental goal to tamper with its reward process, and describe design principles that prevent instrumental (...) goals for two different types of reward tampering. Combined, the design principles can prevent reward tampering from being an instrumental goal. The analysis benefits from causal influence diagrams to provide intuitive yet precise formalizations. (shrink)

In philosophy, the textbook case for the discussion of human motivation is the examination (and almost always, the refutation) of psychological egoism. The arguments have become part of the folklore of our tribe, from their inclusion in countless introductory texts. [...] One of my central aims has been to define the issues empirically, so we do not just settle them by definition. Although I am inclined at present to put my bets on the reward-event theory, with its internalism, monism, and (...) causal primacy of satisfaction, I think we are very far from knowing enough to settle these questions concerning motivation, human or otherwise. The winds of science will blow where they may. In the meantime, we can be a bit more circumspect about what we put in our tribal folklore. (shrink)

The brain and emotion treats emotions as states elicited by reinforcers (reward or punishment), but it is unclear how this view can do justice to the diversity of emotions. It is also unclear how such a view distinguishes emotions from states such as hunger and thirst. A complementary approach to understanding emotions may begin by considering emotions as aspects of social cognition.

People differ in the extent to which they discount the values of future rewards. Behavioural economists measure these differences in terms of functions that describe rates of reduced valuation in the future – temporal discounting – as these vary with time. They measure differences in preference for risk – differing rates of probability discounting – in terms of similar functions that describe reduced valuation of rewards as the probability of their delivery falls. So-called ‘impulsive’ people, including people disposed (...) to addiction, tend to choose rewards in patterns that are described by unusually steeply sloped discount functions. The empirical literature is ambivalent as to whether this applies to pathological gamblers (PGs) who manifest most other behavioural patterns associated with addiction, with different, equally careful studies suggesting opposite conclusions (Petry & Casarella 1999; Holt, Green & Myerson 2003). This puzzle may arise from the fact that most previous experiments on discounting behaviour were not designed so as to allow effects of high temporal discounting to be distinguished from effects of low probability discounting. (Addiction has sometimes been associated with the latter because it seems to involve, at least in the starting addict, under appreciation of risk.) Our research project investigates both forms of discounting behaviour and their relationship to severity of risk of PG in a community sample of South African gamblers. (shrink)

Recent philosophical work has praised the reward structure of science, while recent empirical work has shown that many scientific results may not be reproducible. I argue that the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work, thus contributing to the so-called reproducibility crisis. I use a rational choice model to identify a set of sufficient conditions for this problem to arise, and I argue that these conditions plausibly (...) apply to a wide range of research situations. Currently proposed solutions will not fully address this problem. Philosophical commentators should temper their optimism about the reward structure of science. (shrink)

Research on academic pay finds an unexplained gender pay gap that has not fully dissolved over time and that appears to increase with years of experience. In this study, I consider how role congruity bias contributes to this pay gap. Bias is more likely to manifest in a context where there is some ambiguity about performance and where stereotypes are stronger. I predict that bias in the attribution of credit for coauthored research leads to lower returns to research for female (...) professors. To test this prediction, I use a sample of Canadian faculty in accounting, where research is typically coauthored, where females are underrepresented at the most senior ranks, and where many universities evaluate merit in research, teaching, and service to determine salary increases. In regressions of salary on individual and institutional determinants of salary, I find that women earn marginally less than men. However, the pay gap is only evident for women who publish in a selective list of journals; for the subset of faculty with no publications from this list, there are no significant differences in salary. For researching faculty, the pay gap relates specifically to research productivity. While women publish less on average than men, the returns to their research are also lower. In particular, the relation between the individual’s research ranking and salary is significantly lower for women who publish a higher proportion of their work with men, than for all other faculty. Additional analysis of salary and coauthor patterns confirms that women receive significantly less credit for coauthored articles they publish with men than those they publish with other women but that no similar variations in reward are evident for men across publishing patterns. These findings suggest bias in the attribution of credit for coauthored research in the determination of salary, consistent with role congruity theory, and provide an important potential explanation for why salaries for women vary systematically from those of men even after considering productivity. Providing lower rewards for equal work represents a continuing ethical issue in academe and compounds the challenges women already facing in the profession. (shrink)

Based on brain imaging findings, we present a model according to which addiction emerges as an imbalance in the information processing and integration among various brain circuits and functions. The dysfunctions reflect (a) decreased sensitivity of reward circuits, (b) enhanced sensitivity of memory circuits to conditioned expectations to drugs and drug cues, stress reactivity, and (c) negative mood, and a weakened control circuit. Although initial experimentation with a drug of abuse is largely a voluntary behavior, continued drug use can eventually (...) impair neuronal circuits in the brain that are involved in free will, turning drug use into an automatic compulsive behavior. The ability of addictive drugs to co‐opt neurotransmitter signals between neurons (including dopamine, glutamate, and GABA) modifies the function of different neuronal circuits, which begin to falter at different stages of an addiction trajectory. Upon exposure to the drug, drug cues or stress this results in unrestrained hyperactivation of the motivation/drive circuit that results in the compulsive drug intake that characterizes addiction. (shrink)

Past studies have shown that the perceived time of actions is retrospectively influenced by post-action events. The current study examined whether rewarding performance feedback altered the reported time of action. In Experiment 1, participants performed a speeded button press task and received monetary reward for a presumed “fast,” or a monetary punishment for a presumed “slow” response. Rewarded trials resulted in the false perception that the response action occurred earlier than punished trials. In Experiments 2 and 3, the need for (...) a speeded response and reward were independently manipulated in order to decouple the cognitive and reward components in the feedback signal. When tested independently, neither variable affected the judged time of action. We conclude that meaningful feedback is only used when made salient by reward, to modulate the judged time of an action. (shrink)

A developing neurobiological/psychological theory of positive motivation gives a key causal role to reward events in the brain which can be directly activated by electrical stimulation (ESB). In its strongest form, this Reward Event Theory (RET) claims that all positive motivation, primary and learned, is functionally dependent on these reward events. Some of the empirical evidence is reviewed which either supports or challenges RET. The paper examines the implications of RET for the concepts of 'motivation', 'desire' and 'reward' or 'pleasure'. (...) It is argued (1) that a 'causal base' as opposed to a functional' concept of motivation has theoretical advantages; (2) that a causal distinction between the focus' and the 'anchor' of desire suggests an ineliminable 'opacity' of desire; and (3) that some affective concept, such as 'pleasure', should play a key role in psychological explanation, distinct from that of motivational (or cognitive) concepts. A concept of 'reward' or 'pleasure' as intrinsically positive affect is defended, and contrasted with the more 'operational' definitions of 'reward' in some of the hypotheses of Roy Wise. (shrink)

Managers and others have long debated the merits of different reward systems, such as piecework, hourly rates, bonuses, stock options, and the like. They have usually focused on the efficiency of these systems, but they have also had to consider their side effects on relationships, trust, and calls for fair treatment. Such debates local to every organisation play out the issues of rewards and equity in market-based societies as a whole. This paper examines the concept of equity in the (...) distribution of resources in a society of capitalists. It begins with a discussion of the nature of individual capitalism in modern societies. Then, using production and consumption as the two basic functions of a society, it presents a schema for analysing equity. It concludes with a suggestion for overcoming income inequity. (shrink)

Miller and Rodgers (2001) proposed a central nervous system based Ontogenetic Bonding System that operates across the life course to promote succorant, 1 affiliative, sexual, and nurturant bonds. I discuss features of this theoretical framework that can inform Depue & Morrone-Strupinsky's (D&M-S's) model. Most important, I suggest that the affiliative reward processes D&M-S describe are better conceptualized as subserving the affect/motivation of affection. Footnotes1 “Succorance” is a term coined by Murray (1938) to describe a general tendency to seek the help (...) and protection of others. (shrink)

One of the challenges in the motivation literature is examining the simultaneous effect of different motivational mechanisms on overall motivation and performance. The motivational congruence theory addresses this by stipulating that different motivational mechanisms can reinforce each other if they have similar effects on the perceived locus of causality. Reward salience and choice are two motivational mechanisms which their joint effects have been long debated. Built upon the motivational congruence effect, a recent empirical study affirms that a salient reward in (...) a condition characterized by lack of choice and a non-salient reward in a condition characterized by provision of choice both increase overall motivation and performance. In this study, we examine the effect of reward salience and choice on overall motivation and performance in a controlling context, an effect which has not been studied before. A 2 × 3 factorial design was conducted to examine research hypotheses. The results show that under controlling conditions, salient reward improves overall motivation and performance compared to non-salient and no-reward conditions. (shrink)

Reward and punishment sensitivity seem important traits in understanding behavior in general and psychopathology in particular. Though the definitions used for reward and punishment sensitivity differentiate between responsivity and motivation, the measures thus far used to assess these constructs do not. Further, specificity of the type of reward and punishment in questionnaires might result in measurement bias especially when examining the relationship with psychopathology. Therefore, we developed a stimulus-independent multidimensional questionnaire of reward and punishment sensitivity that differentiates between responsivity and (...) motivation. This study addresses the psychometric qualities of this newly developed reward and punishment responsivity and motivation questionnaire. On the basis of exploratory ordinal factor analysis that was used to examine the quality of the initial pool of 39 items, the number of items was reduced to 18. Confirmatory ordinal factor analysis on the remaining items in an independent sample supported a 18-item four-factor model, and showed acceptable to good internal reliability. The relationship between the subscales of the RPRM-Q and often used questionnaires was examined in the combined sample, which showed some first support for the ability of the new questionnaire to differentiate between responsivity and motivation to approach/avoid. The findings indicate that the RPRM-Q might be a helpful instrument to further test the relevance of punishment and reward sensitivity in psychopathology. (shrink)

In the absence of explicit punitive sanctions, why do individuals voluntarily participate in intergroup warfare when doing so incurs a mortality risk? Here we consider the motivation of individuals for participating in warfare. We hypothesize that in addition to other considerations, individuals are incentivized by the possibility of rewards. We test a prediction of this “cultural rewards war-risk hypothesis” with ethnographic literature on warfare in small-scale societies. We find that a greater number of benefits from warfare is associated (...) with a higher rate of death from conflict. This provides preliminary support for the relationship between rewards and participation in warfare. (shrink)

According to the reward-prediction error hypothesis of dopamine, the phasic activity of dopaminergic neurons in the midbrain signals a discrepancy between the predicted and currently experienced reward of a particular event. It can be claimed that this hypothesis is deep, elegant and beautiful, representing one of the largest successes of computational neuroscience. This paper examines this claim, making two contributions to existing literature. First, it draws a comprehensive historical account of the main steps that led to the formulation and subsequent (...) success of the RPEH. Second, in light of this historical account, it explains in which sense the RPEH is explanatory and under which conditions it can be justifiably deemed deeper than the incentive salience hypothesis of dopamine, which is arguably the most prominent contemporary alternative to the RPEH. (shrink)

Bonuses, as incentives or rewards for health -related behavior, feature prominently in German social health insurance. Their goal is centered around promoting personal responsibility, but reducing overall health -care expenditure and enabling competition between sickness funds also play a role. The central position of personal responsibility in German health -care policy is described, and a framework is offered for an analysis of the ethical issues raised by policies seeking to promote responsibility. The framework entails seven tests relating to: solidarity; (...) equality and equity; intrusiveness; attributability and opportunity of choice; evidence, rationale, and feasibility; affected third parties; and coherence. It is contended that a focus on tests, in particular from within contractualism, is the most appropriate way of addressing the question of health responsibility in pluralistic societies. However, taken by themselves, the tests also allow an identification of the key ethical issues by particular policies, which are illustrated by applying the framework to particular bonus polices. (shrink)

While it is known that reward induces attentional prioritization, it is not clear what effect reward-learning has when associated with stimuli that are not fully perceived. The masked priming paradigm has been extensively used to investigate the indirect impact of brief stimuli on response behavior. Interestingly, the effect of masked primes is observed even when participants choose their responses freely. While classical theories assume this process to be automatic, recent studies have provided evidence for attentional modulations of masked priming effects. (...) Most such studies have manipulated bottom-up or top-down modes of attentional selection, but the role of “newer” forms of attentional control such as reward-learning and selection history remains unclear. In two experiments, with number and arrow primes, we examined whether reward-mediated attentional selection modulates masked priming when responses are chosen freely. In both experiments, we observed that primes associated with high-reward lead to enhanced free-choice priming compared to primes associated with no-reward. The effect was seen on both proportion of choices and response times, and was more evident in the faster responses. In the slower responses, the effect was diminished. Our study adds to the growing literature showing the susceptibility of masked priming to factors related to attention and executive control. (shrink)

Prominent mathematician William Thurston was praised by other mathematicians for his intellectual generosity. But what does it mean to say Thurston was intellectually generous? And is being intellectually generous beneficial? To answer these questions I turn to virtue epistemology and, in particular, Roberts and Wood's (2007) analysis of intellectual generosity. By appealing to Thurston's own writings and interviewing mathematicians who knew and worked with him, I argue that Roberts and Wood's analysis nicely captures the sense in which he was intellectually (...) generous. I then argue that intellectual generosity is beneficial because it counteracts negative effects of the reward structure of mathematics that can stymie mathematical progress. (shrink)

Research has shown that temporary task goals capture more attention than negative, threatening cues, even in anxious individuals. In the current study, we investigated whether temporary task goals...