Abstract not found

The recently developed ‘two-step ’ behavioural task promises to differentiate model-based from model-free reinforcement learning, while generating neurophysiologically-friendly decision datasets with parametric variation of decision variables. These desirable features have prompted its widespread adoption. Here, we analyse the interactions between a range of different strategies and the structure of transitions and outcomes in order to examine con-straints on what can be learned from behavioural performance. The task involves a trade-off between the need for stochasticity, to allow strategies to be discriminated, and a need for determinism, so that it is worth subjects ’ investment of effort to exploit the contingencies optimally. We show through simulation that under certain conditions model-free strategies can masquerade as being model-based. We first show that seemingly innocuous modifica-tions to the task structure can induce correlations between action values at the start of the trial and the subsequent trial events in such a way that analysis based on comparing suc-cessive trials can lead to erroneous conclusions. We confirm the power of a suggested cor-rection to the analysis that can alleviate this problem. We then consider model-free