In this paper we address the problem of finding the simulated system with the best (maximum or minimum) expected performance when the number of alternatives is finite, but large enough that ranking-and-selection (R&S) procedures may require too much computation to be practical. Our approach is to use the data provided by the first stage of sampling in an R&S procedure to screen out alternatives that are not competitive and thereby avoid the (typically much larger) second-stage sample for these systems. Our procedures represent a compromise between standard R&S procedures---that are easy to implement, but can be computationally inefficient---and fully sequential procedures---that can be statistically efficient, but are more difficult to implement and depend on more restrictive assumptions. We present a general theory for constructing combined screening and indifference-zone selection procedures, several specific procedures and a portion of an extensive empirical evaluation. ...

Although most scheduling problems are NP-hard, domain specific techniques perform well in practice but are quite expensive to construct. In adaptive problem-solving, domain specific knowledge is acquired automatically for a general problem solver with a flexible control architecture. In this approach, a learning system explores a space of possible heuristic methods for one well-suited to the eccentricities of the given domain and problem distribution. In this article, we discuss an application of the approach to scheduling satellite communications. Using problem distributions based on actual mission requirements, our approach identifies strategies that not only decrease the amount of CPU time required to produce schedules, but also increase the percentage of problems that are solvable within computational resource limitations. 1. Introduction With the maturation of automated problem-solving research has come grudging abandonment of the search for "the" domain-independent problem solve...

This work is motivated by the need to solve the inventory routing problem when implementing a business practice called vendor managed inventory replenishment (VMI). With VMI, vendors monitor their customers ’ inventories, and decide when and how much inventory should be replenished at each customer. The inventory routing problem attempts to coordinate inventory replenishment and transportation in such a way that the cost is minimized over the long run. We formulate a Markov decision process model of the stochastic inventory routing problem, and propose approximation methods to find good solutions with reasonable computational effort. We indicate how the proposed approach can be used for other Markov decision processes involving the control of multiple resources. ∗ Supported by the National Science Foundation under grant DMI-9875400.

We consider the problem of comparing a finite number of stochastic systems with respect to a single system (designated as the "standard") via simulation experiments. The comparison is based on expected performance, and the goal is to determine if any system has larger expected performance than the standard, and if so to identify the best of the alternatives. In this paper we provide two-stage experiment design and analysis procedures to solve the problem for a variety of scenarios, including when we encounter unequal variances across systems, and when we use the variance reduction technique of common random numbers and it is appropriate to do so. The emphasis is added because in some cases common random numbers can be counterproductive when performing comparisons with a standard. We also provide methods for estimating the critical constants required by our procedures, present a portion of an extensive empirical study and demonstrate one of the procedures via a numerical example. 1 Intr...

this paper, the results naturally apply to (asymptotically) stationary time series.

This paper provides an advanced tutorial on the construction of ranking-and-selection procedures for selecting the best simulated system. We emphasize procedures that provide a guaranteed probability of correct selection, and the key theoretical results that are used to derive them.

In this paper I advocate a new model for inductive learning. Called sequential induction, this model bridges classical fixed-sample learning techniques (which are efficient but ad hoc), and worst-case approaches (which provide strong statistical guarantees but are too inefficient for practical use). According to the sequential inductive model, learning is a sequence of decisions which are informed by training data. By analyzing induction at the level of these decisions, and by utilizing the minimum data necessary to make each decision, sequential inductive techniques can provide the strong statistical guarantees of worst-case methods, but with substantially less data than those methods require. The sequential inductive model is also useful as a method for determining a sufficient sample size for inductive learning and as such, is relevant to megainduction,where the preponderance of data introduces problems of scale. The peepholing and decision-theoretic subsampling approaches of Catlet...

We present a new technique for adaptively choosing the sequence of molecular compounds to test in drug discovery. Beginning with a base compound, we consider the problem of searching for a chemical derivative of the molecule that best treats a given disease. The problem of choosing molecules to test to maximize the expected quality of the best compound discovered may be formulated mathematically as a ranking-andselection problem in which each molecule is an alternative. We apply a recently developed algorithm, known as the knowledge-gradient algorithm, that uses correlations in our Bayesian prior distribution between the performance of different alternatives (molecules) to dramatically reduce the number of molecular tests required, but it has heavy computational requirements that limit the number of possible alternatives to a few thousand. We develop computational improvements that allow the knowledge-gradient method to consider much larger sets of alternatives, and we demonstrate the method on a problem with 87,120 alternatives.

Discrete-event simulation models are often constructed so that an analyst may compare two or more competing design alternatives. This paper presents a survey of the literature for two widely-used statistical methods for selecting the best design from among a finite set of k alternatives: ranking and selection (R&S) and multiple comparison procedures (MCPs). A comprehensive survey of each topic is presented along with a summary of recent unified R&S-MCP approaches. In addition, an example of the application of Nelson and Matejcik’s (1995) combined R&S-MCP procedure is given.

Ranking and selection procedures (R&S) were developed by statisticians to search for the best among a small collection of populations or treatments, where the “best ” treatment is typically the one with the largest or smallest expected (long-run average) response. R&S procedures have been successfully extended to address situations that are encountered in stochastic simulation of alternative system designs, including unequal variances across alternatives; dependence both within the output of each system and across the outputs from alternative systems; and large numbers of alternatives to compare. In nearly all cases the estimator of the expected response is a (perhaps generalized) sample mean of the output of interest. In this paper we derive R&S procedures that employ control-variate estimators instead of sample means. Control variates (CVs) can be much more statistically efficient than sample means, leading to R&S procedures that are correspondingly more efficient. We also consider the related problem of estimating the expected value of the best (as opposed to the selected) system design. 1