## Selecting The Best System: A Decision-Theoretic Approach (1997)

Venue: | In Proc. 1997 Winter Simulation Conference |

Citations: | 11 - 2 self |

### BibTeX

@INPROCEEDINGS{Chick97selectingthe,

author = {Stephen Chick},

title = {Selecting The Best System: A Decision-Theoretic Approach},

booktitle = {In Proc. 1997 Winter Simulation Conference},

year = {1997},

pages = {326--333},

publisher = {IEEE}

}

### OpenURL

### Abstract

The problem of selecting the best system from a finite set of alternatives is considered from a Bayesian decision-theoretic perspective. The framework presented is quite general, and permits selection from two or more systems, with replications that use either independent or common random numbers, with unknown means and covariances for the output, and permits Gaussian or non-Gaussian simulation output. For the case of unknown mean and variance with common random numbers, the framework provides a probability of correct selection that does not suffer from problems associated with the Bonferroni inequality. We indicate some criteria for which the Bayesian approach and other approaches are in general agreement, or disagreement. The probability of correct selection can be calculated either by quadrature or by Monte Carlo simulation from the posterior distribution of the parameters of the statistical distribution of the simulation output. We also comment on expected-value decision-making ver...

### Citations

1454 |
Simulation modeling and analysis
- Law, Kelton
- 1991
(Show Context)
Citation Context ...t system, discrete optimization, decision-theory, Bayesian statistics. 1 MOTIVATION One important application for stochastic simulation is the selection of the best system from a set of alternatives (=-=Law and Kelton, 1991-=-). Goldsman and Nelson (Goldsman and Nelson, 1994) provide a comprehensive literature review of ranking, selection, and Submitted 31 March 1997 for the 1997 Winter Simulation Conference. Updated: 22 M... |

1149 |
Bayesian theory
- Bernardo, Smith
- 1994
(Show Context)
Citation Context ...ions, the sample K-variant mean �� o and a samplesK \Theta K covariance S, where �� o = R X r=1 o r R S = R X r=1 (o r \Gamma �� o)(o r \Gamma �� o) t : (7) The posterior distribution =-=for ��; ~AE is (Bernardo and Smith, 1994) p (�� j �� o; S) �� WK (ff 0 + R=2-=-; W 1 (��o; S)) p (~AE j \Sigma; �� o; S) �� NK ` n 0 �� 0 +R��o n 0 +R ; \Sigma n 0 + R ' where W 1 (��o; S) \Gamma1 = fi 0 + 1 2 S + n 0 R(��o \Gamma �� 0 )(��o \... |

806 | Optimal Statistical Decisions - DeGroot - 1970 |

115 |
Testing a point null hypothesis: the irreconcilability of p-values and evidence (with comments
- Berger, Sellke
- 1987
(Show Context)
Citation Context ...he P-value provides a measure of evidence which is roughly parallel to a Bayesian posterior distribution. If these three conditions are not satisfied, or if the point hypothesis testing framework of (=-=Berger and Sellke, 1987-=-) were used, the P-value would not not be an appropriate measure of evidence for the reasons detailed in (Berger and Sellke, 1987) and briefly summarized in Sec. 9. 6 COMMENTS ON THE BEM Alternate sel... |

46 |
Reconciling Bayesian and frequentist evidence in the one-sided testing problem (with discussion
- Casella, Berger
- 1987
(Show Context)
Citation Context ...second, regardless of the prior distribution p \Theta i jH1 true (` i ) chosen. The relation between posterior probability and P-value of a hypothesis can take on different forms for different tests (=-=Casella and Berger, 1987-=-). In the example of Sec. 5, 1 \Gamma P-value=2 and p (AE 2sAE 1 j D) were close. More generally, for K = 2 systems, the P-value and the probability of correct selection (using the framework above) ar... |

22 |
Two-stage multiple comparisons with the best for computer simulation
- Matejcik, Nelson
- 1995
(Show Context)
Citation Context ...t distribution is shown to play a role for the Bayesian framework of selecting the best system when a Gaussian approximation for the output and a normal-Wishart distribution for the prior are taken. (=-=Matejcik and Nelson, 1995-=-) anticipated this for the special case of unknown means and a known covariance. 5 EXAMPLE A well-known example (Law and Kelton, 1991) of using common random numbers to compare two systems is the Zipp... |

20 | Bayesian analysis for simulation input and output
- Chick
- 1997
(Show Context)
Citation Context ...hers (see Sec. 9). The present work provides an alternate probabilistic measure of evidence that extends a recently proposed Bayesian framework for analyzing the simulation output of a single system (=-=Chick, 1997-=-). The relevant assumptions required from that paper are given in Sec. 2. The selection problem for independent simulation replications is covered in Sec. 3. An extension to handle dependencies such a... |

15 |
A single-sample multipledecision procedure for selecting the multinomial event which has the highest probability
- Bechhofer, Elmaghraby, et al.
- 1959
(Show Context)
Citation Context ...briefly summarized in Sec. 9. 6 COMMENTS ON THE BEM Alternate selection processes exist in the literature. One is the BEM multinomial selection procedure described by Bechofer, Elmaghraby, and Morse (=-=Bechhofer et al., 1959-=-). Miller, Nelson and Reilly (Miller et al., 1996) describe an extension of BEM, the All Vector Comparison (AVC), along with analysis, simulations, and conjectures. The BEM can be used to estimate the... |

12 |
Ranking, selection, and multiple comparison in computer simulation
- Goldsman, Nelson
- 1994
(Show Context)
Citation Context ...eory, Bayesian statistics. 1 MOTIVATION One important application for stochastic simulation is the selection of the best system from a set of alternatives (Law and Kelton, 1991). Goldsman and Nelson (=-=Goldsman and Nelson, 1994-=-) provide a comprehensive literature review of ranking, selection, and Submitted 31 March 1997 for the 1997 Winter Simulation Conference. Updated: 22 May 1997. comparison techniques for selecting the ... |

9 |
The transient behavior of the M/M/s queue, with implications for steady-state simulation
- Kelton, Law
- 1985
(Show Context)
Citation Context ...al time is assumed to be 1, and both have the same utilization,sae = 0:9. Utility ~u in this case is taken to be the negative of average delay in queue of the first 100 customers. Analytical results (=-=Kelton and Law, 1985-=-) indicate that the theoretical expected utilities are AE 1 = \Gamma4:13 and AE 2 = \Gamma3:70. We simulate to evaluate the performance of the results in Sec. 4.2 in determining that the Klunkytel has... |

5 | Applying Bayesian ideas in simulation - Andradóttir, Bier - 1996 |

3 |
A Bayesian batch means methodology for analysis of simulation output
- Andrews, Schriber
- 1983
(Show Context)
Citation Context ...t �� k has a gamma prior distributionsGa (ff k ; fi k ), and that AE k , given oe 2 k = �� \Gamma1 k , has Gaussian conditional distribution N 1 \Gamma �� 0k ; oe 2 k =n 0k \Delta . This g=-=eneralizes (Andrews and Schriber, 1983; Andrad-=- 'ottir and Bier, 1996), where the variance is assumed known. Then the posterior distribution of the expected (conditional) utility has Student distribution p \Upsilon kjD (AE k ) �� St 1 / ��... |

3 |
Getting more from the data in a multinomial selection problem
- Miller, Nelson, et al.
- 1996
(Show Context)
Citation Context ...Alternate selection processes exist in the literature. One is the BEM multinomial selection procedure described by Bechofer, Elmaghraby, and Morse (Bechhofer et al., 1959). Miller, Nelson and Reilly (=-=Miller et al., 1996-=-) describe an extension of BEM, the All Vector Comparison (AVC), along with analysis, simulations, and conjectures. The BEM can be used to estimate the probability that a given system is best in cases... |