). In this context, we present an adaptive procedure for play — called “modified-regret-matching ” — which is interpretable as a stimulus-response or reinforcement procedure, and which has the property that any limit point of the empirical distribution of play is a correlated equilibrium of the stage game. 1

. The decomposition, known as the MAXQ decomposition, has both a procedural semantics---as a subroutine hierarchy---and a declarative semantics---as a representation of the value function of a hierarchical policy. MAXQ unifies and extends previous work on hierarchical reinforcement learning by Singh, Kaelbling

Abstract Function approximation is essential to reinforcement learning, but the standard approach of approximating a value function and determining a policy from it has so far proven theoretically intractable. In this paper we explore an alternative approach in which the policy is explicitly

reinforcement procedures

. The results at the end of a year indicated a significant increase in the performance of reinforced "desirable " behaviors and a general improvement in patient initiative, responsibility, and social interaction. Although investigators may disagree as to what specific strategies or tactics to pursue

In Pavlovian conditioning, animals learn to associate initially neutral stimuli with positive or negative outcomes, leading to appetitive and aversive learning respectively. The honeybee (Apis mellifera) is a prominent invertebrate model for studying both versions of olfactory learning and for unraveling the influence of genotype. As a queen bee mates with about 15 males, her worker offspring belong to as many, genetically-different patrilines. While the genetic dependency of appetitive learning is well established in bees, it is not the case for aversive learning, as a robust protocol was only developed recently. In the original conditioning of the sting extension response (SER), bees learn to associate an odor (conditioned stimulus-CS) with an electric shock (unconditioned stimulus- US). This US is however not a natural stimulus for bees, which may represent a potential caveat for dissecting the genetics underlying aversive learning. We thus first tested heat as a potential new US for SER conditioning. We show that thermal stimulation of several sensory structures on the bee’s body triggers the SER, in a temperature-dependent manner. Moreover, heat applied to the antennae, mouthparts or legs is an efficient US for SER conditioning. Then, using microsatellite analysis, we analyzed heat sensitivity and aversive learning performances in ten worker patrilines issued from a naturally inseminated queen. We demonstrate a strong influence of genotype on aversive learning, possibly indicating the existence of a genetic determinism of this capacity. Such determinism could be instrumental for efficient task partitioning within the hive.

, the requirement for rein-forcement in this procedure is the emission of a mini-mum number of pecks to each of the keys. The pigeon receives food when it completes the requirement on both keys. The frequency of responding to each key was a close approximation to the minimum re-quirement. The present experiment

that deals with this problem by specifying that certain procedures cause a conditioned stimulus (CS) to lose effectiveness; in particular, we argue that a CS will lose associability when its consequences are accurately predicted. In contrast to other current models, the effectiveness of the reinforcer

To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson’s disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require

This paper presents a new approach to hierarchical reinforcement learning based on the MAXQ decomposition of the value function. The MAXQ decomposition has both a procedural semantics---as a subroutine hierarchy---and a declarative semantics---as a representation of the value function of a