A scheme is described for locally Bayesian parameter updating in models structured as successions of component functions. The essential idea is to back-propagate the target data to interior modules, such that an interior component’s target is the input to the next component that maximizes the probability of the next component’s target. Each layer then does locally Bayesian learning. The approach assumes online trial-by-trial learning. The resulting parameter updating is not globally Bayesian but can better capture human behavior. The approach is implemented for an associative learning model that first maps inputs to attentionally filtered inputs and then maps attentionally filtered inputs to outputs. The Bayesian updating allows the associative model to exhibit retrospective revaluation effects such as backward blocking and unovershadowing, which have been challenging for associative learning models. The back-propagation of target values to attention allows the model to show trial-order effects, including highlighting and differences in magnitude of forward and backward blocking, which have been challenging for Bayesian learning models.

Associative learning theories assume that cue interaction and, specifically, retrospective revaluation occur only when the target cue is previously trained in compound with the to-be-revalued cue. However, there are recent demonstrations of retrospective revaluation in the absence of compound training (e.g., Matute & Pineño, 1998a, 1998b). Nevertheless, it seems reasonable to assume that cue interaction should be stronger when the cues are trained together than when they are trained apart. In two experiments with humans, we directly compared compound and elemental training of cues. The results showed that retrospective revaluation in the elemental condition can be as strong as and, sometimes, stronger than that in the compound condition. This suggests that within-compound associations are not necessary for retrospective revaluation to occur and that these effects can possibly be best understood in the framework of general interference theory. In the literature of animal conditioning and human associative learning, it is well known that if a cue, X, is consistently followed by an outcome, O (i.e., X–O), X is generally learned as a predictor of the occurrence of the outcome. It is also well known that responding to X in a subsequent test phase becomes altered if another cue, A, is trained in compound with X as a predictor of the same outcome. Some classic instances of these cue interaction effects in the animal learning literature are overshadowing (Pavlov, 1927), blocking (Kamin, 1968), conditioned inhibition (Pavlov, 1927), and the relative stimulus validity

Standard connectionist models of pattern completion like an auto-associator, typically fill in the activation of a missing feature with internal input from nodes that are connected to it. However, associative studies on competition between alternative causes, demonstrate that people do not always complete the activation of a missing feature, but rather actively encode it as missing whenever its presence was highly expected. Dickinson and Burke's revaluation hypothesis [4] predicts that there is always forward competition of a novel cause, but that backward competition of a known cause depends on a consistent (positive) relation with the alternative cause. This hypothesis was confirmed in several experiments. These effects cannot be explained by standard auto-associative networks, but can be accounted for by a modified auto-associative network that is able to recognize absent information as missing and provides it with negative, rather than positive activation from related nodes. 1.

Within-compound associations in retrospective revaluation and in direct learning: A challenge for comparator theory

Reversal from blocking in humans as a result of posttraining extinction of the blocking stimulus FRANCISCO ARCEDIANO, MARTHA ESCOBAR, and HELENA MATUTE Universidad de Deusto, Bilbao, Spain In a blocking procedure, conditioned stimulus (CS) A is paired with the unconditioned stimulus (US) in Phase 1, and a compound of CSs A and X is then paired with the US in Phase 2. The usual result of such a treatment is that X elicits less conditioned responding than if the A–US pairings of Phase 1 had not occurred. Obtaining blocking with human participants has proven difficult, especially if a behavioral task is used or if the control group experiences reinforcement of a CS different from the blocking CS in Phase 1. In the present series, in which human participants and a behavioral measure of learning were used, we provide evidence of blocking, using the above described control condition. Most important, we demonstrate that extinction of the blocking CS (A) following blocking treatment reverses the blocking deficit (i.e., increases responding to X). These results are at odds with traditional associative theories of learning, but they support current associative theories that predict that posttraining manipulations of the competing stimulus can result in a reversal of stimulus competition phenomena.

What is the value of an action that has never been tried before? One way to frame this question is as an inductive problem: how can I generalize my previous experience with one set of actions to a novel action? We show how hierarchical Bayesian inference can be used to solve this problem, and describe an equivalence between the Bayesian model and temporal difference learning algorithms that have been proposed as models of human reinforcement learning. In two experiments we test several predictions of this model, providing behavioral evidence that humans learn and exploit structured inductive knowledge to make predictions about novel actions. We suggest a new interpretation of dopaminergic responses to novelty in light of this model. Keywords: reinforcement learning, Bayesian inference, exploration, exploitation

Backward blocking (BB) and interference between cues (IbC) are cue competition effects produced by very similar manipulations. In a standard BB design, both effects might occur simultaneously, which implies a potential problem for studying BB. In the present study with humans, the magnitude of both effects was compared using a non-causal scenario and a within-subjects design. Previous studies have made this comparison using learning tasks framed within causal scenarios. This posits a limit to generalizing their findings to non-causal learning situations because there is ample evidence showing that participants engage in causal reasoning when tasks are causally framed. The results obtained showed BB and IbC effects of the same magnitude in a non– causally-framed task. This highlights the methodological need for an IbC control in BB experiments.