## Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization (2000)

Venue: | Journal of Discrete Event Dynamic Systems: Theory and Applications |

Citations: | 56 - 17 self |

### BibTeX

@ARTICLE{Chen00simulationbudget,

author = {Chun-hung Chen and Jianwu Lin and Stephen E. Chick},

title = {Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization},

journal = {Journal of Discrete Event Dynamic Systems: Theory and Applications},

year = {2000},

pages = {251--270}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract. Ordinal Optimization has emerged as an efficient technique for simulation and optimization. Exponential convergence rates can be achieved in many cases. In this paper, we present a new approach that can further enhance the efficiency of ordinal optimization. Our approach determines a highly efficient number of simulation replications or samples and significantly reduces the total simulation cost. We also compare several different allocation procedures, including a popular two-stage procedure in simulation literature. Numerical testing shows that our approach is much more efficient than all compared methods. The results further indicate that our approach can obtain a speedup factor of higher than 20 above and beyond the speedup achieved by the use of ordinal optimization for a 210-design example.

### Citations

1246 |
Simulation Modeling and Analysis
- Law, Kelton
(Show Context)
Citation Context ... satisfy the assumptions of analytical models. While DES simulation has many advantages for modeling complex systems, efficiency is still a significant concern when conducting simulation experiments (=-=Law and Kelton, 1991-=-). To obtain a good statistical estimate for a design decision, a large number of simulation samples or replications is usually required for each design alternative. This is due to the slow convergenc... |

1039 |
Bayesian Theory
- Bernardo, Smith
- 1994
(Show Context)
Citation Context ... with a small number of designs. Recently, Chen (1996) introduced an estimation technique that approximates P{CS} for ordinal comparison when the number of designs is large based on a Bayesian model (=-=Bernardo and Smith, 1994-=-). This technique has the added benefit of providing sensitivity information that is useful in solving problem (2). We will incorporate this technique within our budget allocation approach. Many perfo... |

691 | Optimal statistical decisions - DeGroot - 1970 |

104 |
Handbook of Simulation
- Banks
- 1998
(Show Context)
Citation Context ...e roughly the same. To solve problem (2), we must be able to estimate P{CS}. There exists a large literature on assessing P{CS} based on classical statistical models (e.g., Goldsman and Nelson, 1994; =-=Banks, 1998-=- give an excellent survey on available approaches). However, most of these approaches are only suitable for problems with a small number of designs. Recently, Chen (1996) introduced an estimation tech... |

79 |
Design and analysis of experiments for statistical selection, screening, and multiple comparisons. New York:Wiley
- Bechhofer, Santner, et al.
(Show Context)
Citation Context ... be too small as the estimates of the mean and the variance may be very poor, resulting in premature termination of the comparison. A suitable choice for n0 is between 5 and 20 (Law and Kelton, 1991; =-=Bechhofer et al., 1995-=-). Also, a large � can result in waste of computation time to obtain an unnecessarily high confidence level. On the other hand, if � is small, we need to the computation procedure in step 2 many times... |

63 | On two-stage selection procedures and related probability-inequalities - Rinott - 1978 |

44 |
Stochastic Approximation for Constrained and Unconstrained Systems, volume 26 of Applied Mathematical Sciences
- Kushner, Clark
- 1978
(Show Context)
Citation Context ...ssed as a confidence interval) of this estimator cannot improve faster than O(1/ √ N), the result of averaging i.i.d. noise, where N is the number of simulation samples or replications (Fabian, 1971; =-=Kushner and Clark, 1978-=-). If the accuracy requirement is high, and if the total number of designs in a decision problem is large, then the total simulation cost can easily become prohibitively high. Ordinal Optimization has... |

34 | An Effective Approach to Smartly Allocate Computing Budget for Discrete Event Simulation - Chen - 1995 |

32 | A Guide to Simulation. 2nd ed - Bratley, Fox, et al. - 1987 |

30 |
A lower bound for the correct subset-selection probability and its application to discrete-event system simulations
- Chen
- 1996
(Show Context)
Citation Context ... (APCS). APCS can be computed very easily and quickly; no extra Monte Carlo simulation is needed. Numerical tests show that the APCS approximation can still lead to highly efficient procedures (e.g., =-=Chen, 1996-=-; Inoue and Chick, 1998). We therefore use APCS to approximate P{CS} as the simulation experiment proceeds. More specifically, we consider the following problem: max N1,...,Nk s.t. 1 − k∑ i=1,i̸=b { }... |

28 |
Stochastic comparison algorithm for discrete optimization with estimation
- Gong, Ho, et al.
- 1992
(Show Context)
Citation Context ...ood design. Further Dai (1996) shows that the convergence rate for ordinal optimization can be exponential. This idea has been successfully applied to several problems (e.g., Cassandras et al., 1998; =-=Gong et al., 1999-=-; Patsis et al., 1997). While ordinal optimization could significantly reduce the computational cost for DES simulation, there is potential to further improve its performance by intelligently controll... |

27 | Computing efforts allocation for ordinal optimization and discrete event simulation - Chen, Chen, et al. - 2000 |

24 | New Development of Optimal Computing Budget Allocation for Discrete Event Simulation - Chen, Chen, et al. - 1997 |

24 | HOrdinal Optimization of DEDS
- Ho, Vakili
- 1992
(Show Context)
Citation Context ... emerged as an efficient technique for simulation and optimization. The underlying philosophy is to obtain good estimates through ordinal comparison while the value of an estimate is still very poor (=-=Ho et al., 1992-=-). If our goal is to find the good designs rather than to find an accurate estimate of the best performance value, which is true in many practical situations, it is advantageous to use ordinal compari... |

21 | Convergence Properties of Ordinal Comparison in the Simulation of Discrete Event Dynamic Systems - Dai - 1996 |

20 | Bayesian Analysis for Simulation Input and Output
- Chick
- 1997
(Show Context)
Citation Context ...llocation procedure. Efficiency is more crucial than estimation accuracy in this setting. We adopt a common approximation procedure used in simulation and statistics literature (Brately et al., 1987; =-=Chick, 1997-=-; Law and Kelton, 1991). This approximation is based on the Bonferroni inequality. Let Yi be a random variable. According to the Bonferroni inequality, P{∩k i=1 (Yi < 0)} ≥ 1 − ∑k i=1 [1 − P{Yi < 0}].... |

17 | Design and Analysis of Experiments for - Bechhofer, Santner, et al. - 1995 |

13 |
A table for Rinott’s selection procedure
- Wilcox
- 1984
(Show Context)
Citation Context ...i h2 /d 2 ) ⌉−n0 , for i = 1, 2,...,k, where ⌈•⌉ is the integer “round-up” function, d is the indifference zone, h is a constant which solves Rinott’s integral (h can also be found from the tables in =-=Wilcox, 1984-=-). In short, the computing budget is allocated proportionally to the estimated sample variances.262 CHEN ET AL. Figure 1. P{CS} vs. T using five different allocation procedures for experiment 1. Norm... |

12 |
Ranking, selection and multiple comparisons in computer simultaions
- Goldsman, Nelson
- 1994
(Show Context)
Citation Context ...es for different designs are roughly the same. To solve problem (2), we must be able to estimate P{CS}. There exists a large literature on assessing P{CS} based on classical statistical models (e.g., =-=Goldsman and Nelson, 1994-=-; Banks, 1998 give an excellent survey on available approaches). However, most of these approaches are only suitable for problems with a small number of designs. Recently, Chen (1996) introduced an es... |

11 | An approximation approach of the standard-clock method for general discrete-event simulation - Chen, Ho - 1995 |

10 | Ordinal Comparison of Heuristic Algorithms Using Stochastic Optimization - Chen, Wu, et al. - 1999 |

8 | Comparison of Bayesian and frequentist assessments of uncertainty for selecting the best system
- Inoue, Chick
- 1998
(Show Context)
Citation Context ...S can be computed very easily and quickly; no extra Monte Carlo simulation is needed. Numerical tests show that the APCS approximation can still lead to highly efficient procedures (e.g., Chen, 1996; =-=Inoue and Chick, 1998-=-). We therefore use APCS to approximate P{CS} as the simulation experiment proceeds. More specifically, we consider the following problem: max N1,...,Nk s.t. 1 − k∑ i=1,i̸=b { } P ˜Jb > ˜Ji k∑ Ni = T ... |

7 | Introduction - Harmon, Walker, et al. - 2001 |

6 |
SIMD Parallel Discrete Event Dynamic System Simulation
- Patsis, Chen, et al.
- 1997
(Show Context)
Citation Context ... Dai (1996) shows that the convergence rate for ordinal optimization can be exponential. This idea has been successfully applied to several problems (e.g., Cassandras et al., 1998; Gong et al., 1999; =-=Patsis et al., 1997-=-). While ordinal optimization could significantly reduce the computational cost for DES simulation, there is potential to further improve its performance by intelligently controlling the simulation ex... |

5 |
Stochastic approximation,” in Optimizing Methods in Statistics
- Fabian
- 1971
(Show Context)
Citation Context ...ypically expressed as a confidence interval) of this estimator cannot improve faster than O(1/ √ N), the result of averaging i.i.d. noise, where N is the number of simulation samples or replications (=-=Fabian, 1971-=-; Kushner and Clark, 1978). If the accuracy requirement is high, and if the total number of designs in a decision problem is large, then the total simulation cost can easily become prohibitively high.... |

3 | Optimization Algorithm with Probabilistic Estimation - Yan, Mukai - 1993 |

3 |
Introduction to Mathematical Programming
- Walker
- 1999
(Show Context)
Citation Context ...,i is defined in section 2). Then, let F be the Lagrangian relaxation of (5): k∑ ∫ ∞ 1 F = 1 − √ e 2π t2 ( ) k∑ 2 dt − λ Ni − T . i=1 i̸=b − δ b,i σ b,i i=1 Furthermore, the Karush-Kuhn-Tucker (KKT) (=-=Walker, 1999-=-) conditions of this problem can be stated as follows. ∂ F ∂ F = ( ∂ Ni ∂ − δb,i ( ∂ − ) σb,i δb,i ) σb,i ∂σb,i − λ ∂σb,i ∂ Ni = −1 2 √ 2π exp [ 2 −δb,i 2σ 2 ] δb,iσ b,i 2 i N 2 i (σ 2 − λ = 0, for i ... |

1 |
Ordinal Optimization for Deterministic and Stochastic Discrete Resource Allocation
- Cassandras, Dai, et al.
- 1998
(Show Context)
Citation Context ...cting252 CHEN ET AL. a good design. Further Dai (1996) shows that the convergence rate for ordinal optimization can be exponential. This idea has been successfully applied to several problems (e.g., =-=Cassandras et al., 1998-=-; Gong et al., 1999; Patsis et al., 1997). While ordinal optimization could significantly reduce the computational cost for DES simulation, there is potential to further improve its performance by int... |