This paper provides an advanced tutorial on the construction of ranking-and-selection procedures for selecting the best simulated system. We emphasize procedures that provide a guaranteed probability of correct selection, and the key theoretical results that are used to derive them.

It is known that Hough Transform computation can be significantly accelerated by polling instead of voting. A small part of the data set is selected at random and used as input to the algorithm. The performance of these Probabilistic Hough Transforms depends on the poll size. Most Probabilistic Hough algorithms use a fixed poll size, which is far from optimal since conservative design requires the fixed poll size to be much larger than necessary in average conditions. It has recently been experimentally demonstrated that adaptive termination of voting can lead to improved performance in terms of the error rate versus average poll size tradeoff. However, the lack of a solid theoretical foundation made general performance evaluation and optimal design of adaptive stopping rules nearly impossible.

The problem of selecting the best system from a finite set of alternatives is considered from a Bayesian decision-theoretic perspective. The framework presented is quite general, and permits selection from two or more systems, with replications that use either independent or common random numbers, with unknown means and covariances for the output, and permits Gaussian or non-Gaussian simulation output. For the case of unknown mean and variance with common random numbers, the framework provides a probability of correct selection that does not suffer from problems associated with the Bonferroni inequality. We indicate some criteria for which the Bayesian approach and other approaches are in general agreement, or disagreement. The probability of correct selection can be calculated either by quadrature or by Monte Carlo simulation from the posterior distribution of the parameters of the statistical distribution of the simulation output. We also comment on expected-value decision-making ver...

Abstract not found

The purpose of this paper is (1) to provide a theoretical justification for the use of Monte-Carlo sampling for approximate resolution of NP-hard maximization problems in the framework of weighted parsing, and (2) to show how such sampling techniques can be e#ciently implemented with an explicit control of the error probability. We provide an algorithm to compute the local sampling probability distribution that guarantee that the global sampling probability indeed corresponds to the aimed theoretical score. The proposed sampling strategy significantly di#ers from existing methods, showing by the same way the bias induced by these methods.

Ingcreasingly, data-mining algorithms must deal with databases that continuously grow over time. These algorithms must avoid repeatedly scanning their databases. When database attributes are symbolic, ADtrees have already shown to be efficient structures to store sufficient statistics in main memory and to accelerate the mining process in batch environments. Here we present an efficient method to sequentially update ADtrees that is suitable for incremental environments. 1.

Consider a stochastic simulation experiment that generates v independent vector replications consisting of an observation from each of k systems. Typical system comparisons are based on mean (long-run) performance. However, the probability that a system will actually be the best is sometimes more relevant, and can provide a very different perspective than the systems' means. Empirically, we select one system as the best performer (i.e., it wins) on each replication. Each system has an unknown constant probability of winning on any replication and the numbers of wins for the individual systems follow a multinomial distribution. Procedures exist for finding the system with the largest probability of being the best. This paper addresses the companion problem of estimating the probability that each system will be the best. The maximum likelihood estimators (MLEs) of the multinomial cell probabilities for a set of v vector replications across k systems are well known. We use these same v ve...