This paper explicates the interaction between the implicit and explicit learning processes in skill acquisition, contrary to the common tendency in the literature of studying each type of learning in isolation. It highlights the interaction between the two types of processes and its various effects on learning, including the synergy effect. This work advocates an integrated model of skill learning that takes into account both implicit and explicit processes; moreover, it embodies a bottom-up approach (first learning implicit knowledge and then explicit knowledge on its basis) towards skill learning. The paper shows that this approach accounts for various effects in the process control task data, in addition to accounting for other data reported elsewhere.

In developing autonomous agents, one usually emphasizes only (situated) procedural knowledge, ignoring more explicit declarative knowledge. On the other hand, in developing symbolic reasoning models, one usually emphasizes only declarative knowledge, ignoring procedural knowledge. In contrast, we have developed a learning model Clarion, which is a hybrid connectionist model consisting of both localist and distributed representations, based on the two-level approach proposed in Sun (1995). Clarion learns and utilizes both procedural and declarative knowledge, tapping into the synergy of the two types of processes, and enables an agent to learn in situated contexts and generalize resulting knowledge to different scenarios. It unifies connectionist, reinforcement, and symbolic learning in a synergistic way, to perform on-line, bottom-up learning. This summary paper presents one version of the architecture and some results of the experiments. Key Words: hybrid models, sequential decision ...

Abstract not found

To further explore the interaction between the implicit and explicit learning processes in skill acquisition (which have been tackled before, e.g., in Sun et al 2001, 2005), this paper explores details of the interaction of different learning modes: implicit learning, explicit hypothesis testing learning, and implicit-to-explicit knowledge extraction. Contrary to the common tendency in the literature to study each type of learning in isolation, this paper highlights the interaction among them and various effects of the interaction on learning, including the synergy effect. This work advocates an integrated model of skill learning that takes into account both implicit and explicit learning processes; moreover, it also uniquely embodies a bottom-up (implicit-to-explicit) learning approach in addition to other types of learning. The paper shows that this model accounts for various effects in the human behavioral data from the psychological experiments with the process control task, in addition to accounting for other data in other psychological experiments (which has been reported elsewhere). The paper shows that to account for these effects, implicit learning, bottom-up implicit-to-explicit extraction, and explicit hypothesis testing learning are all needed.

This paper studies the interaction between implicit and explicit processes in skill learning, in terms of top-down learning (that is, learning that goes from explicit to implicit knowledge) vs. bottom-up learning (that is, learning that goes from implicit to explicit knowledge). Instead of studying each type of knowledge (implicit or explicit) in isolation, we highlight the interaction between the two types of processes, especially in terms of one type giving rise to another. The work presents an integrated model of skill learning that takes into account both implicit and explicit processes and both top-down and bottom-up learning. We examine and simulate human data in the Tower of Hanoi task. The paper shows how the quantitative data in this task may be captured using either top-down or bottom-up approaches, although top-down learning is a more apt explanation of the human data currently available.

Several dierent ways of using symbolic methods to enhance reinforcement learning are identied and discussed in some detail. Each demonstrates to some extent the potential advantages of combining RL and symbolic methods. Dierent from existing work, in combining RL and symbolic methods, we focus on autonomous learning from scratch without a priori domain-specic knowledge. Thus the role of symbolic methods lies truly in enhancing learning, not in providing a priori domain-specic knowledge. These discussed methods point to the possibilities and the challenges in this line of research. 1

Abstract: Despite many promising results from the use of reinforcement learning in simulated robot worlds, its use in real robot worlds is relatively rare. This paper addresses challenges related to real robot worlds and shows how reinforcement learning combined with linear function approximation can solve many of them. Experiments are performed using a light-seeking robot built with the Lego Mindstorms Robotics Invention System. Key-Words:- reinforcement learning, exploration strategies, light-seeking robot, linear approximation, gradient descent 1

This paper is concerned with knowledge extraction from reinforcement learners. It addresses two approaches towards knowledge extraction: the extraction of explicit, symbolic rules from neural reinforcement learners, and the extraction of complete plans from such learners. The advantages of such knowledge extraction include (1) the improvement of learning (especially with the rule extraction approach), and (2) the improvement of the usability of results of learning. I. Introduction This paper is concerned with knowledge extraction from reinforcement learners (especially when they are implemented in neural networks, which serve as function approximators), in sequential decision making tasks. It covers work by me and my collaborators over the past 4 years (see Sun 1997, Sun and Peterson 1998, 1999, Sun and Sessions 1998a, b). Specifically, it addresses two approaches towards knowledge extraction: the extraction of explicit, symbolic rules from neural reinforcement learners, and the extra...

Sequence learning is arguably the most prevalent form of human and animal learning. Sequences play a pivotal role in classical studies of instrumental conditioning,[1] in human skill learning, and in human high-level problem solving and reasoning. So, it's logical that sequence learning is an important component of learning in many task domains of intelligent systems: inference, planning, reasoning, robotics, natural language processing...

| This paper will discuss learning in hybrid models that goes beyond simple rule extraction from backpropagation networks. Although simple rule extraction has received a lot of research attention, to further develop hybrid learning models that include both symbolic and subsymbolic knowledge and that learn autonomously, it is necessary to study autonomous learning of both subsymbolic and symbolic knowledge in integrated architectures. This paper will describe knowledge extraction from neural reinforcement learning. It includes two approaches towards extracting plan knowledge: the extraction of explicit, symbolic rules from neural reinforcement learning, and the extraction of complete plans. This work points to the creation of a general framework for achieving the subsymbolic to symbolic transition in an integrated autonomous learning framework. 1 Introduction This paper will discuss learning in hybrid models that goes beyond simple rule extraction from backpropagation networks. Althou...