We are interested in the problem faced byanagent with limited computational capabilities, embedded in a complex environment with other agents and processes not under its control. Careful management of computational resources is important for complex problem-solving tasks in which the time spent in decision making affects the quality of the responses generated by a system.

In this paper we outline a general approach to the study of metareasoning, not in the sense of explicating the semantics of explicitly specified meta-level control policies, but in the sense of providing a basis for selecting and justifying computational actions. This research contributes to a developing attack on the problem of resource-bounded rationality, by providing a means for analysing and generating optimal computational strategies. Because reasoning about a computation without doing it necessarily involves uncertainty as to its outcome, probability and decision theory will be our main tools. We develop a general formula for the utility of computations, this utility being derived directly from the ability of computations to affect an agent's external actions. We address some philosophical difficulties that arise in specifying this formula, given our assumption of limited rationality. We also describe a methodology for applying the theory to particular problem-solving systems, a...

Partially observable Markov decision processes (POMDPs) can be used to model complex control problems that include both action outcome uncertainty and imperfect observability. A control problem within the POMDP framework is expressed as a dynamic optimization problem with a value function that combines costs or rewards from multiple steps. Although the POMDP framework is more expressive than other simpler frameworks, like Markov decision processes (MDP), its associated optimization methods are more demanding computationally and only very small problems can be solved exactly in practice. Our work focuses on two possible approaches that can be used to solve larger problems: approximation methods and exploitation of additional problem structure. First, a number of new eÆcient approximation methods and improvements of existing algorithms are proposed. These include (1) the fast informed bound method based on approximate dynamic programming updates that lead to piecewise linear and convex v...

Controlling a robot involves making decisions that modify its behavior. Making good decisions may require time-consuming computation. Changes in the environment over time affect when this computation can be done (e.g., after obtaining the necessary information) , and when a result is useful (e.g., before some event occurs). This sensitivity to when computation is performed and when decisions are made is what makes these problems "time-dependent." A controller with more than one decision to make must trade off computation time, based on the expected effect on the system's behavior. We call the resulting meta-level scheduling problem a "deliberation-scheduling" problem. We have

Major military operations and other large-scale activities naturally involve competitions for resources that must be managed effectively to ensure overall success. Since plan revisions may change the resource demands of the activity, the relevant competitions for resources involve computational resources (time, database control, etc.) as well as the more obvious non-computational resources (fuel, aircraft, etc.). Ideally, these conflicting demands should be resolved rationally in the sense of decision theory and economics. We determine rational allocations over the full range of resources by using an artificial market economy (implemented as RECON, the Reasoning ECONomy) to determine prices or trading ratios among resources. We represent tasks and goals as resource-endowed consumers and computational methods, informational resources, and reasoning procedures as resource-transforming producers. We then use the information provided by such problem-oriented market economies to guide searc...

Efficiency dictates that plans for large-scale distributed activities be revised incrementally, with parts of plans being revised only if the expected utility of identifying and revising the subplans improve on the expected utility of using the original plan. The problems of identifying and reconsidering the subplans affected by changed circumstances or goals are closely related to the problems of revising beliefs as new or changed information is gained. But the current techniques of reason maintenance--- the standard method for belief revision---choose revisions arbitrarily and enforce global notions of consistency and groundedness which may mean reconsidering all beliefs or plan elements at each step. We outline revision methods that revise only those beliefs and plans worth revising, and that tolerate incoherence and ungroundedness when these are judged less detrimental than a costly revision effort. 1 Introduction Planning is necessary for the organization of large-scale activitie...

To perform rational decision-making, autonomous agents need considerable computational resources. In multi-agent settings, when other agents are present in the environment, these demands are even more severe. We investigate ways in which the agent's knowledge and the results of deliberative decision-making can be compiled to reduce the complexity of decision-making procedures and to save time in urgent situations. We use machine learning algorithms to compile decision-theoretic deliberations into condition-action rules on how to coordinate in a multi-agent environment. Using different learning algorithms, we endow a resource-bounded agent with a tapestry of decision making tools, ranging from purely reactive to fully deliberative ones. The agent can then select a method depending on the time constraints of the particular situation. We also propose combining the decision-making tools, so that, for example, more reactive methods serve as a pre-processing stage to the more accurate but sl...

We present efficient techniques for computing near optimal strategies for a class of stochastic commodity trading problems modeled as Markov decision processes (MDPs). The process has a continuous state space and a large action space and cannot be solved efficiently by standard dynamic programming methods. We exploit structural properties of the process, and combine it with Monte-Carlo estimation techniques to obtain novel and efficient algorithms that closely approximate the optimal strategies. 1

The ability to categorize and use concepts e#ectively is a basic requirementofany intelligent actor. The utility-based approach to categorization is founded on the thesis that categorization is fundamentally in service of action, i.e., the choice of concepts made by an actor is critical to its choice of appropriate actions. This is in contrast to classical and similarity-based approaches which seek logical completeness in concept description with respect to sensory data rather than action-oriented e#ectiveness. Utility-based categorization is normative and not descriptive. It prescribes howanintelligent agent ought to conceptualize to act e#ectively. It provides ideals for categorization, speci#es criteria for the design of e#ective computational agents, and provides a model of ideal competence. A decision-theoretic framework for utilitybased categorization whichinvolves reasoning about alternative categorization models of varying levels of abstraction is proposed. Categorization mode...

of the Dissertation Incremental Search Methods for Real-Time Decision Making by Joseph Carl Pemberton Doctor of Philosophy in Computer Science University of California, Los Angeles, 1995 Professor Richard Korf, Chair Many real-world problems, such as air-traffic control and factory scheduling, require that a sequence of decisions be made in real time, without complete information. Since there is typically not sufficient time for traditional methods to find a complete solution before committing to a decision, we propose an incremental search method for making real-time decisions. Our approach is to separate the real-time decision task into three sub-problems: where to spend limited computational resources?, when to stop computing?, and how to make decisions given incomplete information? By interleaving computation with execution, we can use the execution time to improve the solution quality. We present the last incremental decision problem as a simplification of the general increme...