## Sensitivity in risk analyses with uncertain numbers (2006)

Citations: | 7 - 0 self |

### BibTeX

@MISC{Ferson06sensitivityin,

author = {Scott Ferson and W. Troy Tucker},

title = {Sensitivity in risk analyses with uncertain numbers},

year = {2006}

}

### OpenURL

### Abstract

Sensitivity analysis is a study of how changes in the inputs to a model influence the results of the model. Many techniques have recently been proposed for use when the model is probabilistic. This report considers the related problem of sensitivity analysis when the model includes uncertain numbers that can involve both aleatory and epistemic uncertainty and the method of calculation is Dempster-Shafer evidence theory or probability bounds analysis. Some traditional methods for sensitivity analysis generalize directly for use with uncertain numbers, but, in some respects, sensitivity analysis for these analyses differs from traditional deterministic or probabilistic sensitivity analyses. A case study of a dike reliability assessment illustrates several methods of sensitivity analysis, including traditional probabilistic assessment, local derivatives, and a “pinching ” strategy that hypothetically reduces the epistemic uncertainty or aleatory uncertainty, or both, in an input variable to estimate the reduction of uncertainty in the outputs. The prospects for applying the methods to black box models are also considered. 3

### Citations

878 |
Interval analysis
- Moore
- 1966
(Show Context)
Citation Context ...inistic calculation, when the model involves uncertainty about the real-valued quantities used in the calculation, uncertainty analysis can be conducted via interval analysis (Young 1931; Dwyer 1951; =-=Moore 1966-=-; Alefield and Herzberger 1983; Neumaier 1990). Probability theory, implemented perhaps by Monte Carlo simulation, can also be used as an uncertainty analysis of a deterministic calculation because it... |

828 |
Fuzzy Sets and Fuzzy Logic Theory and Applications. Upper Saddle River
- Klir, Yuan
- 1995
(Show Context)
Citation Context ...tions of the p-box enclose all possible distributions in the same way that the endpoints of the interval circumscribe the possible real values. Dempster-Shafer evidence theory (Oberkampf et al. 2005; =-=Klir and Yuan 1995-=-; Shafer 1976) has been widely studied in computer science and artificial intelligence, although it has never achieved wide acceptance among probabilists and traditional statisticians. In a discrete p... |

758 |
Applied logistic regression
- Hosmer, Lemeshow
- 2000
(Show Context)
Citation Context ...nse surface methods (Myers 1971; Morton 1983; Downing et al. 1985; Kleijnen 1992; Myers 1999; Myers et al. 2004), the Fourier amplitude sensitivity test (Cukier et al. 1978), and logistic regression (=-=Hosmer and Lemeshow 1989-=-). The last of these can be used to directly estimate sensitivities of probabilities (McCarthy et al. 1995). Finally, the calculus approaches include manually engineered symbolic differentiation (Iman... |

742 |
Statistical Reasoning with Imprecise Probabilities
- Walley
- 1991
(Show Context)
Citation Context ...an be easy to compute, but getting those bounds optimally tight given available information can sometimes be difficult. PBA and DST are practical approaches to computing with imprecise probabilities (=-=Walley 1991-=-). As in a Bayesian sensitivity analysis, imprecise probabilities are represented by a class of distribution functions. PBA is simpler than the general theory of imprecise probabilities because it def... |

658 |
An introduction to copulas
- Nelsen
- 2006
(Show Context)
Citation Context ...ty (Ferson et al. 2004). The simulation achieved the smallest reduction in uncertainty when the relationship between A and B was described by an intermediate dependence function described by Clayton (=-=Nelsen 1999-=-; Ferson et al. 2004) with correlation coefficient r = −0.64. Notice that the range of variation in percent reduction of uncertainty is often small or zero, but that it can also be rather large. For i... |

508 |
Interval methods for systems of equations
- Neumaier
- 1990
(Show Context)
Citation Context ...es uncertainty about the real-valued quantities used in the calculation, uncertainty analysis can be conducted via interval analysis (Young 1931; Dwyer 1951; Moore 1966; Alefield and Herzberger 1983; =-=Neumaier 1990-=-). Probability theory, implemented perhaps by Monte Carlo simulation, can also be used as an uncertainty analysis of a deterministic calculation because it yields a distribution describing the probabi... |

455 | Introduction to Interval Computations - Alefeld, Herzberger - 1983 |

370 |
Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis
- Morgan, Henrion
- 1990
(Show Context)
Citation Context ...used as an uncertainty analysis of a deterministic calculation because it yields a distribution describing the probability of alternative possible values about a point estimate (Iman and Helton 1988; =-=Morgan and Henrion 1990-=-; Helton and Davis 2000b; 2003). In the figure below these two possible paths are shown as right and left downward arrows respectively. Second-order probability distributions Probability distributions... |

295 | Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation - Griewank - 2000 |

273 | Applied Interval Analysis - Jaulin, Kieffer, et al. - 2001 |

209 | Sensitivity Analysis - Saltelli, Chan, et al. - 2000 |

185 | Computational Complexity and Feasibility of Data Processing and Interval Computations - Kreinovich, Lakeyev, et al. - 1997 |

156 |
Response Surface Methodology
- Myers, Montgomery
- 1995
(Show Context)
Citation Context ...s. This approach works if the evaluation cost is moderate, but it often isn’t for the probabilistic models encountered in complex analyses. The regression approaches include response surface methods (=-=Myers 1971-=-; Morton 1983; Downing et al. 1985; Kleijnen 1992; Myers 1999; Myers et al. 2004), the Fourier amplitude sensitivity test (Cukier et al. 1978), and logistic regression (Hosmer and Lemeshow 1989). The ... |

145 |
Tchebycheff Systems with Applications in Analysis and Statistics
- Karlin, Studden
- 1966
(Show Context)
Citation Context ... structures. However, this generality might be possible by devising a scheme for “sampling” discrete probability distributions from a non-parametric uncertain number. The theory of Chebyshev systems (=-=Karlin and Studden 1966-=-) suggests that the bounds on uncertain numbers are set by degenerate discrete probability distributions having mass only on a minimal number of points. It should be possible to generate example distr... |

141 | Discrete event systems: Sensitivity analysis and stochastic optimization via the score function method - Rubinstein, Shapiro - 1993 |

139 | On automatic differentiation
- Griewank
- 1989
(Show Context)
Citation Context ...lculus approaches include manually engineered symbolic differentiation (Iman and Helton 1988), computer calculus (Oblow 1983a; 1983b), and automatic differentiation (Fischer 1993; Korivi et al. 1994; =-=Griewank 1989-=-; 2000). Only the first is immediately applicable to probabilistic models, but it can involve many person-months of effort (Iman and Helton 1988). General approaches for sensitivity analysis of discre... |

136 | Gradient estimation via perturbation analysis - Glasserman - 1991 |

124 |
R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing
- Team
- 2008
(Show Context)
Citation Context ...sitivities were computed semi-analytically, without any Monte Carlo sampling, using the RAMAS Risk Calc software (Ferson 2002) and specially prepared scripts for the R environment (R Development Core =-=Team 2004-=-). The results are displayed in Figure 3. Because these estimates contain both epistemic and aleatory uncertainty, they cannot be fully characterized by scalar numbers. Even their summary statistics s... |

104 | Perturbation Analysis and Discrete Event Dynamic Systems - Ho, Cao - 1991 |

90 | Sensitivity estimates for nonlinear mathematical models - Sobol’ - 1990 |

72 | Probabilistic Arithmetic I: Numerical Methods for Calculating - Williamson, Downs - 1990 |

68 | Computational Methods of Linear Algebra - Faddeev, Faddeeva - 1963 |

68 | Sensitivity analysis in practice: A guide to assessing scientific models - Saltelli, Tarantola, et al. - 2004 |

67 |
Constructing probability boxes and Dempster-Shafer structures
- Ferson, Kreinovich, et al.
- 2003
(Show Context)
Citation Context ...ply bounding arguments to the probabilistic calculation and arrive at interval versions of probability distributions. We call such calculations “probability bounds analysis” (PBA) (Ferson 1994; 2002; =-=Ferson et al. 2003-=-). This approach represents the uncertainty about a probability distribution by the set of cumulative distribution functions lying entirely within a pair of bounding distribution functions called a “p... |

67 | Optimization via simulation: a review - Fu - 1994 |

67 |
A Distribution-Free Approach to Inducing Rank Correlation among Input Variables
- Iman, Conover
- 1982
(Show Context)
Citation Context ...rate size commonly encountered in risk analysis, the computations for DST and PBA can be faster than a numerically intensive sensitivity analysis with traditional methods (Iman et al. 1980; 1981a,b;; =-=Iman and Conover 1982-=-; Iman and Helton 1988; Morgan and Henrion 1990; Saltelli et al. 2000; Helton and Davis 2000b). The advantages outlined above of the new methods are limited to models involving explicitly known calcul... |

63 |
C.: An investigation of uncertainty and sensitivity analysis techniques for computer models
- Iman, Helton
- 1988
(Show Context)
Citation Context ...alyses and there are many disparate approaches based on various measures of influence and response (Helton et al. 2006b; Cacuci 2003; Frey and Patil 2002; Frey 2002; Saltelli et al. 2000; Hamby 1994; =-=Helton 1993-=-; Iman and Helton 1988; and references therein). L’Ecuyer (1991) and Fu (1994) provided general reviews on the estimation of gradients in simulation studies. Cacuci et al. (2003; 2005) have promoted t... |

61 |
A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output
- Saltelli, Tarantola, et al.
- 1999
(Show Context)
Citation Context ...tivity analysis is a study of how the variation in the output of a model can be apportioned to different sources of variation in the inputs (e.g., Saltelli 2003, Li et al. 2001; Rabitz and Alis 1999; =-=Saltelli et al. 1999-=-; Sobol’ 1993; inter alia). This approach decomposes variance in the output in terms of the contributions from the several inputs. This is surely different from the earlier interpretation of sensitivi... |

57 | The Algebra of Random Variables - Springer - 1979 |

56 | Survey of sampling-based methods for uncertainty and sensitivity - Helton, Johnson, et al. |

51 |
Propagation of Uncertainty in Risk Assessments: The Need to Distinguish Between Uncertainty Due to Lack of Knowledge and Uncertainty Due to Variability.” Risk Analysis
- Hoffman, Hammonds
- 1994
(Show Context)
Citation Context ...risk analyses and safety assessments. One could construct a probabilistic uncertainty analysis of a probabilistic calculation. The resulting analysis would be a second-order probabilistic assessment (=-=Hoffman and Hammonds 1994-=-; Cullen and Frey 1999). However, such studies can be difficult to conduct because of the large 10snumber of calculations that are required. It is also sometimes difficult to visualize the results in ... |

47 | Probabilistic sensitivity analysis of complex models - Oakley, O’Hagan - 2004 |

46 | Uncertainty and Information: Foundations of Generalized Information Theory - Klir - 2006 |

42 | Hypercube Sampling and the Propagation of Uncertainty in - Helton, Davis, et al. - 2003 |

39 | Small Sample Sensitivity Analysis Techniques for Computer Models, with an Application to Risk Assessment - Iman, Conover - 1980 |

36 |
Treatment of Uncertainty in Performance Assessments for Complex Systems.” Risk Analysis
- Helton
- 1994
(Show Context)
Citation Context ...pagated according to the rules of interval analysis because it represents ignorance. The distinction between these two forms of uncertainty is considered very important in practical risk assessments (=-=Helton 1994-=-; Hoffman and Hammonds 1994; Paté-Cornell 1996; Helton 1997). PBA and DST are useful because they can account for the distinction when analysts think it is important, but the methods do not require th... |

35 | Ramas Risk Calc 4.0 Software: Risk Assessment with Uncertain Numbers - Ferson - 2002 |

33 | Making best use of model evaluations to compute sensitivity indices - Saltelli - 2002 |

31 |
Automatically Verified Reasoning with Both Intervals and Probability Density Functions
- Berleant
- 1993
(Show Context)
Citation Context ...ture is the uncertain number characterizing the generalized convolution between the two Dempster-Shafer structures. The algorithm for convolving p-boxes under independence (Williamson and Downs 1990; =-=Berleant 1993-=-; 1996) is essentially identical to Yager’s Cartesian product. The input p-boxes are discretized into Dempster-Shafer structures, the Cartesian product is computed, and the output p-box is reconstitut... |

29 | Automatically verified arithmetic on probability distributions and intervals - Berleant - 1996 |

29 | An Approach to Sensitivity Analysis of Computer Models, Part 2 - Iman, Helton, et al. - 1981 |

29 | Error estimations for indirect measurements: randomized vs. deterministic algorithms for ‘black-box’ programs - Trejo, Kreinovich - 2007 |

26 |
Identification and review of sensitivity analysis methods
- Frey, Patil
- 2002
(Show Context)
Citation Context ... analysis consequently has many manifestations in probabilistic analyses and there are many disparate approaches based on various measures of influence and response (Helton et al. 2006b; Cacuci 2003; =-=Frey and Patil 2002-=-; Frey 2002; Saltelli et al. 2000; Hamby 1994; Helton 1993; Iman and Helton 1988; and references therein). L’Ecuyer (1991) and Fu (1994) provided general reviews on the estimation of gradients in simu... |

26 |
Uncertainty and sensitivity analysis in the presence of stochastic and subjective uncertainty
- Helton
- 1997
(Show Context)
Citation Context ... it represents ignorance. The distinction between these two forms of uncertainty is considered very important in practical risk assessments (Helton 1994; Hoffman and Hammonds 1994; Paté-Cornell 1996; =-=Helton 1997-=-). PBA and DST are useful because they can account for the distinction when analysts think it is important, but the methods do not require the distinction in order to work. The two forms of uncertaint... |

25 |
A comparison of uncertainty and sensitivity analysis results obtained with random and Latin hypercube sampling
- Helton, Davis, et al.
- 2005
(Show Context)
Citation Context ...yst undertakes a “what-if” sensitivity study. Monte Carlo simulations and Bayesian analyses can be viewed as a kind of sensitivity analysis themselves (Helton and Davis 2002; Morgan and Henrion 1990; =-=Iman and Helton 1985-=-) in that they yield a distribution describing the variability about a deterministic point estimate. This section suggests that, likewise, DST and PBA are also kinds of sensitivity analyses when they ... |

25 | An Overview of Derivative Estimation - L'Ecuyer - 1991 |

23 |
Nonlinear Sensitivity Analysis of Multiparameter Model Systems
- Cukier, Levine, et al.
- 1978
(Show Context)
Citation Context ...lyses. The regression approaches include response surface methods (Myers 1971; Morton 1983; Downing et al. 1985; Kleijnen 1992; Myers 1999; Myers et al. 2004), the Fourier amplitude sensitivity test (=-=Cukier et al. 1978-=-), and logistic regression (Hosmer and Lemeshow 1989). The last of these can be used to directly estimate sensitivities of probabilities (McCarthy et al. 1995). Finally, the calculus approaches includ... |

23 |
Arithmetic and other operations on Dempster-Shafer structures
- Yager
- 1986
(Show Context)
Citation Context ...ferent the result could be given all 15sof the professed uncertainty. The requisite computations used in PBA and DST are actually quite simple and have been implemented in straightforward algorithms (=-=Yager 1986-=-; Williamson and Downs 1990; Berleant 1993; 1996; Ferson 2002; Ferson and Hajagos 2004). For a large class of problems of moderate size commonly encountered in risk analysis, the computations for DST ... |

20 |
and second-order aerodynamic sensitivity derivatives via automatic differentiation with incremental iterative methods, A
- Sherman, Taylor, et al.
- 1994
(Show Context)
Citation Context ...95). Finally, the calculus approaches include manually engineered symbolic differentiation (Iman and Helton 1988), computer calculus (Oblow 1983a; 1983b), and automatic differentiation (Fischer 1993; =-=Korivi et al. 1994-=-; Griewank 1989; 2000). Only the first is immediately applicable to probabilistic models, but it can involve many person-months of effort (Iman and Helton 1988). General approaches for sensitivity ana... |

20 | Utilizing belief functions for the estimation of future climate change - Kriegler, Held |