In which we try to give a basic intuitive sense of what reinforcement learning is and how it differs and relates to other fields, e.g., supervised learning and neural networks, genetic algorithms and artificial life, control theory. Intuitively, RL is trial and error (variation and selection, search) plus learning (association, memory). We argue that RL is the only field that seriously addresses the special features of the problem of learning from interaction to achieve long-term goals.

This paper provides a detailed discussion of wireless resource and channel allocation schemes. We provide a survey of a large number of published papers in the area of fixed, dynamic and hybrid allocation schemes and compare their trade-offs in terms of complexity and performance. We also investigate these channel allocation schemes based on other factors such as distributed/centralized control and adaptability to traffic conditions. Moreover, we provide a detailed discussion on reuse partitioning schemes, effect of hand-offs and prioritization schemes. Finally, we discuss other important issues in resource allocation such as overlay cells, frequency planning, and power control. 1 Introduction Technological advances and rapid development of handheld wireless terminals have facilitated the rapid growth of wireless communications and mobile computing. Taking ergonomics and economics factors into account, and considering the new trends in the telecommunications industry to provide ubiqui...

Abstract—The shadow cluster concept can be used to estimate future resource requirements and to perform call admission decisions in wireless networks. Shadow clusters can be used to decide if a new call can be admitted to a wireless network based on its quality-of-service (QoS) requirements and local traffic conditions. The shadow cluster concept can especially be useful in future wireless networks with microcellular architectures where service will be provided to users with diverse QoS requirements. The framework of a shadow cluster system is completely distributed, and can be viewed as a message system where mobile terminals inform the base stations in their neighborhood about their requirements, position, and movement parameters. With this information, base stations predict future demands, reserve resources accordingly, and admit only those mobile terminals which can be supported adequately. The shadow cluster concept involves some processing and communication overheads. These overheads have no effect on wireless resources, but only on the base stations and the underlying wireline network. In this paper, it is shown how base stations determine the probabilities that a mobile terminal will be active in other cells at future times, define and maintain shadow clusters by using probabilistic information on the future position of their mobile terminals with active calls, and predict resource demands based on shadow cluster information. In addition, a call admission algorithm is introduced, which uses current traffic and bandwidth utilization conditions, as well as the amount of resources and maximum allowable “dropping probability ” being requested. Performance results showing the advantages of the shadow cluster concept are also included in the paper. Index Terms — Active mobile probability, call admission, resource allocation, shadow cluster. I.

In cellular telephone systems, an important problem is to dynamically allocate the communication resource (channels) so as to maximize service in a stochastic caller environment. This problem is naturally formulated as a dynamic programming problem and we use a reinforcement learning (RL) method to find dynamic channel allocation policies that are better than previous heuristic solutions. The policies obtained perform well for a broad variety of call traffic patterns. We present results on a large cellular system with approximately 49^49 states.

Handoff is an essential element of cellular communications. Efficient handoff algorithms are a cost-effective way of enhancing the capacity and QoS of cellular systems. This article presents different aspects of handoff and discusses handoff related features of cellular systems. Several system deployment scenarios that dictate specific handoff requirements are illustrated. An account of handoff-related resource management tasks of cellular systems is given. Implementation of the handoff process is explained. Several mechanisms for evaluation of handoff-related system performance are described.

There has been rapid growth in the demand for mobile communications over the past few years. This has led to intensive research and development efforts for complex PCS (personal communication service) networks. Capacity planning and performance modeling is necessary to maintain a high quality of service to the mobile subscriber while minimizing cost to the PCS provider. The need for flexible analysis tools and the high computational requirements of large PCS network simulations make it an excellent candidate for parallel simulation. Here, we describe our experiences in developing two PCS simulation models on a general purpose distributed simulation platform based on the Time Warp mechanism. These models utilize two widely used approaches to simulating PCS networks: (i) the call-initiated and (ii) the portable-initiated models. We discuss design decisions that were made in mapping these models to the Time Warp executive, and characterize the workloads resulting from these models in term...

Abstract-A new scheme that allows cell gateways (base stations) to borrow channels h m adjacent gateways in a cellular communication system is presented. Borrowed channels are used with reduced transmitted power to limit interference with cochannel cells. No channel locking is needed. The scheme, which can be used with various multiple access techniques, permits simple channel control management without requiring global information about channel usage throughout the system. It provides enhanced traffic performance in homogeneous environments and also can be used to relieve spatially localized traffic overloads (tele-traffic ‘‘hot spots”). Co-channel interference analysis shows that the scheme can maintain the same SIR as nonborrowing schemes. Analytical models using multidimensional birth-death processes and decomposition methods are devised to characterize performance. The results which are also validated by simulation indicate that significantly increased traffic capacity can be achieved in comparison with nonborrowing schemes. I.

Efficient allocation of communication channels is crit-ical for the performance of wireless mobile computing systems. The centralized channel allocation algorithms proposed in literature are neither robust, nor scalable. Distributed channel allocation schemes proposed in the past are complicated and require active participation of the mobile nodes. These algorithms are unable to dy-namically adjust to spatial and temporal fluctuations in channel demand. We present a dynamic distributed channel allocation algorithm that can quickly adapt to changes in load distribution. The algorithm described in this paper requires minimal involvement of the mo-bile nodes, thus conserving their limited energy supply. The algorithm is proved to be deadlock free, starvation free and fair. It prevents co-channel interference and is scalable.

Abstract—It is well known that if a stochastic service system (such as a cellular network) is shared by users with different characteristics (such as differing handoff rates or call holding times), the overall system performance can be improved by denial of service requests even when the excess capacity exists. Such selective denial of service based on system state is defined as call admission. A recent paper suggested the use of genetic algorithms (GA’s) to find near-optimal call admission policies for cellular networks. In this paper, we define local call admission policies that make admission decisions based on partial state information. We search for the best local call admission policies for onedimensional (1-D) cellular networks using genetic algorithms and show that the performance of the best local policies is comparable to optima for small systems. We test our algorithm on larger systems and show that the local policies found outperform the maximum packing and best handoff reservation policies for the systems we have considered. We find that the local policies suggested by the Genetic Algorithm search in these cases are double threshold policies. We then find the best double threshold policies by exhaustive search for both 1-D and Manhattan model cellular networks and show that they almost always outperform the best trunk reservation policies for these systems. Index Terms — Call admission, cellular systems, genetic algorithms, Markov decision processes (MDP), reservation policies.

Cellular data and communication networks are usually modeled as graphs with each node representing a base station in a cell in the network, and edges representing geographical adjacency of cells. The problem of channel assignment in such networks can be seen as a graph multicoloring problem. We survey the models, algorithms, and lower bounds for this problem.