Decision making in noisy and changing environments requires a fine balance between exploiting knowledge about good courses of action and exploring the environment in order to improve upon this knowledge. We present an experiment on a restless bandit task in which participants made repeated choices between options for which the average rewards changed over time. Comparing a number of computational models of participants’ behavior in this task, we find evidence that a substantial number of them balanced exploration and exploitation by (...) considering the probability that an option offers the maximum reward out of all the available options. (shrink)

How, and how well, do people switch between exploration and exploitation to search for and accumulate resources? We study the decision processes underlying such exploration/exploitation trade‐offs using a novel card selection task that captures the common situation of searching among multiple resources (e.g., jobs) that can be exploited without depleting. With experience, participants learn to switch appropriately between exploration and exploitation and approach optimal performance. We model participants' behavior on this task with random, threshold, and sampling strategies, and find that (...) a linear decreasing threshold rule best fits participants' results. Further evidence that participants use decreasing threshold‐based strategies comes from reaction time differences between exploration and exploitation; however, participants themselves report non‐decreasing thresholds. Decreasing threshold strategies that “front‐load” exploration and switch quickly to exploitation are particularly effective in resource accumulation tasks, in contrast to optimal stopping problems like the Secretary Problem requiring longer exploration. (shrink)

Most psychological studies investigating the balance between stability and flexibility in decision making use specific restrictions in their scenarios. These restrictions are likely to affect decision process, and it is unclear which of the findings can be transferred to more naturalistic decision contexts that call for a balance between stability and flexibility. Therefore, the present study used a scenario that is inspired by the problem structure found in a particular domain: Adapt/Exchange decisions in modular chemical plants. In this setting, we (...) investigated whether participants engage in a thorough comparison of options and whether they perseverate on their previous choices when decision sequences increasingly favour one or the other option. The results suggest that instead of comparing options, participants used a satisficing strategy, checking whether Adapt was good enough and only computing an Exchange solution when it was not. Sequence effects were found, but their direction was opposite to the choice perseveration predicted from the literature: In sequences initially favouring Adapt, participants started exploring the Exchange option early on, while in sequences initially demanding Exchange, they preferred Adapt as soon as it became possible. The results raise questions about the application of psychological theories to complex decisions between qualitatively different options. (shrink)