Results 1  10
of
85
Statistical selection of the best system
, 2005
"... This tutorial discusses some statistical procedures for selecting the best of a number of competing systems. The term “best” may refer to that simulated system having, say, the largest expected value or the greatest likelihood of yielding a large observation. We describe various procedures for findi ..."
Abstract

Cited by 79 (7 self)
 Add to MetaCart
This tutorial discusses some statistical procedures for selecting the best of a number of competing systems. The term “best” may refer to that simulated system having, say, the largest expected value or the greatest likelihood of yielding a large observation. We describe various procedures for finding the best, some of which assume that the underlying observations arise from competing normal distributions, and some of which are essentially nonparametric in nature. In each case, we comment on how to apply the above procedures for use in simulations.
Comparisons with a Standard in Simulation Experiments
 Management Science
, 1998
"... We consider the problem of comparing a finite number of stochastic systems with respect to a single system (designated as the "standard") via simulation experiments. The comparison is based on expected performance, and the goal is to determine if any system has larger expected performance ..."
Abstract

Cited by 19 (12 self)
 Add to MetaCart
(Show Context)
We consider the problem of comparing a finite number of stochastic systems with respect to a single system (designated as the "standard") via simulation experiments. The comparison is based on expected performance, and the goal is to determine if any system has larger expected performance than the standard, and if so to identify the best of the alternatives. In this paper we provide twostage experiment design and analysis procedures to solve the problem for a variety of scenarios, including when we encounter unequal variances across systems, and when we use the variance reduction technique of common random numbers and it is appropriate to do so. The emphasis is added because in some cases common random numbers can be counterproductive when performing comparisons with a standard. We also provide methods for estimating the critical constants required by our procedures, present a portion of an extensive empirical study and demonstrate one of the procedures via a numerical example. 1 Intr...
TwoStage Stopping Procedures Based On Standardized Time Series
 Management Science
, 1994
"... this paper we will consider functions h : C[0; 1) ! ! which are typically not continuous, and we let D(h) denote the set of points x 2 C[0; 1) at which h is not continuous. Let fX ffl : ffl ? 0g be a family of random elements taking values in C[0; 1); i.e., the X ffl correspond to stochastic process ..."
Abstract

Cited by 16 (7 self)
 Add to MetaCart
this paper we will consider functions h : C[0; 1) ! ! which are typically not continuous, and we let D(h) denote the set of points x 2 C[0; 1) at which h is not continuous. Let fX ffl : ffl ? 0g be a family of random elements taking values in C[0; 1); i.e., the X ffl correspond to stochastic processes with sample paths in C[0; 1). If X is a random element of C[0; 1), then the X ffl are said to converge weakly to X (written X ffl ) X as ffl ! 0) if
Selecting The Best System: Theory And Methods
, 2003
"... This paper provides an advanced tutorial on the construction of rankingandselection procedures for selecting the best simulated system. We emphasize procedures that provide a guaranteed probability of correct selection, and the key theoretical results that are used to derive them. ..."
Abstract

Cited by 14 (0 self)
 Add to MetaCart
This paper provides an advanced tutorial on the construction of rankingandselection procedures for selecting the best simulated system. We emphasize procedures that provide a guaranteed probability of correct selection, and the key theoretical results that are used to derive them.
AN ASYMPTOTIC ALLOCATION FOR SIMULTANEOUS SIMULATION EXPERIMENTS
, 1999
"... In this paper, we consider the allocation of a fixed total number of simulation replications among competing design alternatives in order to (i) identify the best simulated design, (ii) intelligently determine the best simulation run lengths for all simulation experiments, and (iii) significantly re ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
In this paper, we consider the allocation of a fixed total number of simulation replications among competing design alternatives in order to (i) identify the best simulated design, (ii) intelligently determine the best simulation run lengths for all simulation experiments, and (iii) significantly reduce the total computation cost. An asymptotically optimal allocation rule for maximizing a lower bound of the probability of correct selection is presented. Moreover, we illustrate the efficiency of our method with a series of generic numerical experiments. The simulation cost is significantly reduced with our sequential approach.
Learning and Exploitation do not Conflict under Minimax Optimality
"... . We show that adaptive real time dynamic programming extended with the action selection strategy which chooses the best action according to the latest estimate of the cost function yields asymptotically optimal policies within finite time under the minimax optimality criterion. From this it follows ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
. We show that adaptive real time dynamic programming extended with the action selection strategy which chooses the best action according to the latest estimate of the cost function yields asymptotically optimal policies within finite time under the minimax optimality criterion. From this it follows that learning and exploitation do not conflict under this special optimality criterion. We relate this result to learning optimal strategies in repeated twoplayer zerosum deterministic games. Keywords. reinforcement learning, selfoptimizing systems, dynamic games 1 Introduction Reinforcement learning (RL) concerns practical problems related to learning of optimal behaviour in sequential decision tasks. The most popular theoretical framework adopted by RL researchers is that of Markovian Decision Problems (MDPs). One of the main questions in RL is what extent of exploration is needed for a learner so that the price of exploration does not become too demanding. Usually some exploration (e...
Fully sequential indifferencezone selection procedures with variancedependent sampling
 Naval Research Logistics
, 2006
"... Abstract: Fully sequential indifferencezone selection procedures have been proposed in the simulation literature to select the system with the best mean performance from a group of simulated systems. However, the existing sequential indifferencezone procedures allocate an equal number of samples t ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
(Show Context)
Abstract: Fully sequential indifferencezone selection procedures have been proposed in the simulation literature to select the system with the best mean performance from a group of simulated systems. However, the existing sequential indifferencezone procedures allocate an equal number of samples to the two systems in comparison even if their variances are drastically different. In this paper we propose new fully sequential indifferencezone procedures that allocate samples according to the variances. We show that the procedures work better than several existing sequential indifferencezone procedures when variances of the systems
Analysis of Simulation Output
 In Proceedings of the 2003 Winter Simulation Conference
"... We discuss methods for statistically analyzing the output from stochastic discreteevent or Monte Carlo simulations. Terminating and steadystate simulations are considered. 1 ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
We discuss methods for statistically analyzing the output from stochastic discreteevent or Monte Carlo simulations. Terminating and steadystate simulations are considered. 1
Can a twostage procedure enjoy secondorder properties? Statist
, 1996
"... SUMMARY. We first consider the classical fixedwidth confidence interval estimation problem for the mean µ of a normal population whose variance σ2 is unknown, but a particular application scenario guides the experimenter to assume that σ> σL where σL(> 0) is known. The seminal twostage metho ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
SUMMARY. We first consider the classical fixedwidth confidence interval estimation problem for the mean µ of a normal population whose variance σ2 is unknown, but a particular application scenario guides the experimenter to assume that σ> σL where σL(> 0) is known. The seminal twostage methodology of Stein (1945, 1949), originally proposed when σ(> 0) is completely unknown, obviously needs major revisions since we wish to incorporate such added partial information regarding σ in the determination of the final sample size. In the case of completely unknown σ, Stein’s (1945, 1949) twostage procedure is known to enjoy the consistency property, but it is not even firstorder efficient. In the case when σ> σL(> 0), the revised twostage procedure is shown to enjoy all the usual secondorder properties together with the consistency property. As a followup, we include a simulation exercise in the interval estimation scenario. The minimum risk point estimation problem for µ is also discussed briefly in the same light. 1.
Adjusting for unequal variances when comparing means in oneway and twoway effects ANOVA models
 Journal of Educational Statistics
, 1989
"... Numerous papers have shown that the conventional F test is not robust to unequal variances in the oneway fixed effects ANOVA model, and several methods have been proposed for dealing with this problem. Here I describe and compare two methods for handling unequal variances in the twoway fixed effec ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
Numerous papers have shown that the conventional F test is not robust to unequal variances in the oneway fixed effects ANOVA model, and several methods have been proposed for dealing with this problem. Here I describe and compare two methods for handling unequal variances in the twoway fixed effects ANOVA model. One is based on an improved Wilcox (1988) method for the oneway model, which forms the basis for considering this method in the twoway ANOVA model. The other is an extension of James's (1951) second order method. The primary goal in this paper is to describe and compare two methods for testing null hypotheses in a twoway ANOVA in which the usual homogeneity of variance assumption may not be true. The first approach uses an extension of James's (1951) second order method. The second method is based on an improvement of a technique introduced by Wilcox (1988) for the oneway model. The next section reviews relevant results on the oneway model, the third section describes an improvement of Wilcox's solu