The first edition of this book was published in 1992 by Wiley (ISBN 0 471 93609 X). Since this book is now out of print, and to answer the request of several colleagues, the authors have decided to make it available freely on the Web, while retaining the copyright, for the benefit of the scientific community. Copyright Statement This electronic document is in PDF format. One needs Acrobat Reader (available freely for most platforms from the Adobe web site) to benefit from the full interactive machinery: using the package hyperref by Sebastian Rahtz, the table of contents and all LATEX cross-references are automatically converted into clickable hyperlinks, bookmarks are generated automatically, etc.. So, do not hesitate to click on references to equation or section numbers, on items of thetableofcontents and of the index, etc.. One may freely use and print this document for one’s own purpose or even distribute it freely, but not commercially, provided it is distributed in its entirety and without modifications, including this preface and copyright statement. Any use of thecontents should be acknowledged according to the standard scientific practice. The

We propose a model independent procedure for verifying properties of discrete event systems. The dynamics of such systems can be very complex, making them hard to analyze, so we resort to methods based on Monte Carlo simulation and statistical hypothesis testing. The verification is probabilistic in two senses. First, the properties, expressed as CSL formulas, can be probabilistic. Second, the result of the verification is probabilistic, and the probability of error is bounded by two parameters passed to the verification procedure. The verification of properties can be carried out in an anytime manner by starting off with loose error bounds, and gradually tightening these bounds.

. In this paper, we develop mathematical machinery for verifying that a broad class of general state space Markov chains reacts smoothly to certain types of perturbations in the underlying transition structure. Our main result provides conditions under which the stationary probability measure of an ergodic Harris recurrent Markov chain is differentiable in a certain strong sense. The approach is based on likelihood ratio "change-of-measure" arguments, and leads directly to a "likelihood ratio gradient estimator" that can be computed numerically. Keywords: Harris recurrent Markov chain, likelihood ratio, gradient estimation, regeneration. 1 The research of this author was supported by the U. S. Army Research Office under Contract No. DAAL03-91G -0101 and by the National Science Foundation under Contract No. DDM-9101580. 2 This author's research was supported by NSERC-Canada grant No. OGP0110050 and FCAR-Qu'ebec grant No. 93ER1654. 1. Introduction In this paper, we will study the cl...

We explain the main techniques for estimating derivatives by simulation and survey the most recent developments in that area. In particular, we discuss perturbation analysis (PA), likelihood ratios (LR), weak derivatives (WD), finite differences (FD), and many of their variants. We also mention some other approaches. Our discussion emphasizes the relationships between the methods. For that purpose, all of them are presented in the same framework, which is based on L'Ecuyer (1990). 1. INTRODUCTION Simulation is a popular tool for estimating the expected (average) performance measure of a complex stochastic system. Various statistical techniques have been develop ed in that context. Estimating the derivative or sensitivity of such an expectation certainly looks more difficult, but is nevertheless important for many practical applications. For example, let ` be a real-valued (continuous) parameter and suppose that the performance measure of interest depends on ` either directly, or indi...

. Convergence rate results are derived for a stochastic optimization problem where a performance measure is minimized with respect to a vector parameter `. Assuming that a gradient estimator is available and that both the bias and the variance of the estimator are (known) functions of the budget devoted to its computation, the gradient estimator is employed in conjunction with a stochastic approximation (SA) algorithm. Our interest is to figure out how to allocate the total available computational budget to the successive SA iterations. The effort is devoted to solving the asymptotic version of this problem by finding the convergence rate of SA towards the optimizer, first as a function of the number of iterations, and then as a function of the total computational effort. As a result the optimal rate of increase of the computational budget per iteration can be found. Explicit expressions for the case where the computational budget devoted to an iteration is a polynomial in the iteratio...

. This paper considers efficient simulation techniques for estimating steady-state quantities in models of highly dependable computing systems with general component failure and repair time distributions. Earlier approaches in this application setting for steady-state estimation rely on the regenerative method of simulation, which can be used when the failure time distributions are exponentially distributed. However, when the failure times are generally distributed the regenerative structure is lost and a new approach must be taken. The approach we take is to exploit a ratio representation for steady-state quantities in terms of cycles that are no longer independent and identically distributed. A "splitting" technique is used in which importance sampling is used to speed up the simulation of rare system failure events during a cycle, and standard simulation is used to estimate the expected cycle length. Experimental results show that the method is effective in practice. Keywords. Relia...

Currently the semantics of stochastic process algebras are defined using (an extension) of labelled transition systems. This usually results in a semantics based on the interleaving of causally independent actions. The advantage is that the structure of transition systems closely resembles that of Markov chains, enabling the use of standard solution techniques for analytical and numerical performance assessment of formal specifications. The main drawback is that distributions are restricted to be exponential. In [2] we proposed to use a partial-order semantics for stochastic process algebras. This allows the support of non-exponential distributions in the process algebra in a perspicuous way, but the direct resemblance with Markov chains is lost. This paper proposes to exploit discrete-event simulation techniques for analyzing our partial-order model, called stochastic event structures. The key idea is to obtain from event structures so-called (time-homogeneous) generalized semiMarkov ...

Approximating automata are finite-state representations of the sequential inputoutput behaviors of hybrid systems characterized by threshold events that trigger discrete changes in the continuous dynamic equations. Procedures proposed for constructing approximating automata require forward and backward mappings of sets of continuous state trajectories -- mappings which are not available for arbitrary continuous dynamics. This paper develops the foundations for constructing approximating automata automatically for hybrid systems in which the continuous dynamics are defined by convex polytopes in the vector space of the derivatives of the continuous state trajectories. The computations are illustrated for a simple example which also demonstrates the use of approximating automata to solve verification problems that may be intractable using fixed-point computations for linear hybrid automata. 1 Introduction This paper concerns the generation of purely discrete models (finite automata) for...

this paper, we review some of the importance-sampling techniques that have been developed in recent years to e#ciently estimate dependability measures in Markovian and non-Markovian models of highly dependable systems. 1 Acronyms MTTF Mean time to failure. MTBF Mean time between failures. CTMC Continuous-time Markov chain. DTMC Discrete-time Markov chain. GSMP Generalized semi-Markov process. SAVE System AVailability Estimator. CLT Central limit theorem. VRR Variance reduction ratio. TRR Total e#ort reduction ratio. MSDIS Measure-specific dynamic importance sampling. BLBLR Balance over links balanced likelihood ratio. BLBLRC Balance over links balanced likelihood ratio with cuts. 1 INTRODUCTION High dependability requirements of today's critical and/or commercial systems often lead to complicated and costly designs. The ability to predict relevant dependability measures for such complex systems is essential, not only to guarantee hig

Model-based evaluation of reliable distributed and parallel systems is difficult due to the complexity of these systems and the nature of the dependability measures of interest. The complexity creates problems for analytical model solution techniques, and the fact that reliability and availability measures are based on rare events makes traditional simulation methods inefficient. Importance sampling is a well-known technique However, finding an importance sampling strategy that works well in general is a difficult problem. The best strategy for importance sampling depends on the characteristics of the system and the dependability measure of interest. This fact motivated the development of an environment for importance sampling that would support the wide variety of model characteristics and interesting measures. The environment is based on stochastic activity networks, and importance sampling strategies are specified using the new concept of the importance sampling governor. The governor supports dynamic importance sampling strategies by allowing the stochastic elements of the model to be redefined based on the evolution of the simulation. The utility of the new environment is demonstrated by evaluating the unreliability of a highly dependable fault-tolerant unit used in the well-known MARS architecture. The model is non-Markovian, with Weibull distributed failure times and uniformly distributed repair times.