A variety of methods for developing quasi-optimal intelligent control systems using reinforcement learning techniques based on adaptive critics have appeared in recent years. This paper reviews the family of approximate dynamic programming techniques based on adaptive critic methods and introduces a new hybrid critic training method.

Abstract-- We show that a reinforcement learning method, adaptive critic based approximate dynamic programming, can be used to create fuzzy policy managers for adaptive control of a logistic system. Two different architectures are used for the policy manager, a feed forward neural network, and a fuzzy rule base. For both architectures, policy managers are trained that outperform LP and GA derived fixed policies in stochastic and non-stationary demand environments. In all cases the fuzzy system initialized with expert information outperforms the neural network. Index terms-- applications, neural networks, reinforcement learning, genetic algorithms, qualitative reasoning, rule learning 1.

Adaptive critic based approximate dynamic programming techniques are gradient based methods for finding optimal policies for multi-stage decision processes. We believe adaptive critic methods are now developed to the point that they can be applied to the full spectrum of decision and control problems, including inventory control and job scheduling problems of interest in manufacturing. In this paper we illustrate the use one such technique for the development of a policy manager for a distribution system. We compare the performance of both neural net based and fuzzy rule based policy managers with that of LP and GA based fixed policies. Both adaptive critic based soft computing techniques outperform the fixed policies for stationary and nonstationary stochastic demand conditions. 1