## Talking Probabilities: Communicating Probabilistic Information With Words And Numbers (1999)

Venue: | International Journal of Approximate Reasoning |

Citations: | 28 - 5 self |

### BibTeX

@ARTICLE{Renooij99talkingprobabilities:,

author = {Silja Renooij and Cilia Witteman},

title = {Talking Probabilities: Communicating Probabilistic Information With Words And Numbers},

journal = {International Journal of Approximate Reasoning},

year = {1999},

volume = {22},

pages = {169--194}

}

### Years of Citing Articles

### OpenURL

### Abstract

The number of knowledge-based systems that build on Bayesian belief networks is increasing. The construction of such a network however requires a large number of probabilities in numerical form. This is often considered a major obstacle, one of the reasons being that experts are reluctant to provide numerical probabilities. The use of verbal probability expressions as an additional method of eliciting probabilistic information may to some extent remove this obstacle. In this paper, we review studies that address the communication of probabilities in words and/or numbers. We then describe our own experiments concerning the development of a probability scale that contains words as well as numbers. This scale appears to be an aid for researchers and domain experts during the elicitation phase of building a belief network and might help users understand the output of the network.

### Citations

7319 |
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- Pearl
- 1988
(Show Context)
Citation Context ...g probability; Expert systems; Knowledge elicitation; Explanation 1 Introduction Bayesian belief networks, also referred to as probabilistic networks, were first introduced in the late 1980s by Pearl =-=[39]-=-. Since then, an increasing number of successful applications of such networks in di#erent problem domains have been developed, which demonstrates that they have established their position in Artifici... |

1806 |
The Magical Number Seven, Plus or Minus Two: Some Limits of our Capacity for Processing Information
- Miller
- 1956
(Show Context)
Citation Context ...uld be easier on people's cognitive capacities. It is easier to distinguish some seven information categories or expressions clearly than it is to demarcate the meanings of a long list of expressions =-=[33,50]-=-. Other studies have used lists of 18 expressions [8] or 19 [7,22] or 34 in a long list and 14 in a shorter version [5], as few as two [21] or as many as 52 [34]. These lists were compiled by the rese... |

417 |
Modern Multidimensional Scaling: Theory and Applications
- Borg, Groenen
- 1997
(Show Context)
Citation Context ...riment we asked subjects for pairwise comparison, that is: for similarity judgements among all pairs of verbal probability expressions, to uncover the underlying structure of relationships among them =-=[44,4]-=-. For the eight expressions, there were 28 pairs to compare. A similarity judgement was made by scoring each pair of expressions on a 10 cm. line, using as anchors the expressions 'exact same' and 'co... |

136 |
Nonlinear Multivariate Analysis
- Gifi
- 1990
(Show Context)
Citation Context ...s, we then analysed the data with a non-linear principal components technique developed by the University of Leiden PRINCALS, an acronym for principal components analysis by alternating least squares =-=[20]-=-. It may be used for ordinal data. Important for our study, PRINCALS can compute solutions that reduce the orderings of all subjects together to one or more dimensions and indicate the quality (eigenv... |

131 |
Selecting Uncertainty Calculi and Granularity : An Experiment in Trading-off Precision and Complexity
- Bonissone, S
- 1986
(Show Context)
Citation Context ... an aid, both during the elicitation phase of building a belief network and in understanding the output of the network's computations. We are not the first to study this problem. Bonissone and Decker =-=[3]-=- analyzed the use of linguistic terms to establish a certain granularity to facilitate knowledge elicitation in possibilistic reasoning. In the next paragraph, we review other researchers' empirical s... |

90 | Munin—a causal probabilistic network for interpretation of electromyographic findings - Andreassen, Woldbye, et al. - 1987 |

61 | Regression and linear models - Darlington - 1990 |

58 | The sensitivity of belief networks to imprecise probabilities: an experimental investigation
- Pradhan, Henrion, et al.
- 1996
(Show Context)
Citation Context ...uggested that, even if verbal expressions would be less precise than numerical expressions, imprecision of probabilities does not entail a deterioration in the average performance of a belief network =-=[23,40,36]-=-. This suggestion awaits further corroboration. Limiting the number of verbal expressions in a scale might, however, be advisable. The use of some verbal expressions should be avoided, to wit those fo... |

51 | Nonmetric individual differences multidimensional scaling: An alternating least squares method with optimal scaling features - Takane, Young, et al. - 1977 |

49 |
der Gaag. Elicitation of Probabilities for Belief Networks: Combining Qualitative and Quantitative Information
- Druzdzel, van
- 1995
(Show Context)
Citation Context ...d other health workers express and process probabilities in verbal rather than numerical form [32,30]. Physicians rarely reason using numerical probabilities, and if they do, they tend to make errors =-=[14,49]-=-. Kuipers and colleagues [30] conclude that to physicians, subjective probabilities are not numbers. Brun and Teigen pursued their inquiry and discovered what could be called a variation of the commun... |

49 |
Measuring the Vague Meanings of Probability Terms
- Wallsten, Budescu, et al.
- 1986
(Show Context)
Citation Context ... vice versa. In such studies, researchers have found a great between subject variability in the numerical values assigned to verbal probabilities and great overlap between the words (see among others =-=[2,53]-=-). Much less within and between subject variability was found in the numerical probability expressions subjects used when asked to describe a graphical representation of chance than in the verbal expr... |

38 | Converting a rule-base expert system into a belief network
- Korver, Lucas
- 1993
(Show Context)
Citation Context ...orks in di#erent problem domains have been developed, which demonstrates that they have established their position in Artificial Intelligence as valuable representations of reasoning with uncertainty =-=[1,25,28,12,31]-=- . A belief network consists of a qualitative and a quantitative part. The qualitative part is a directed graph, where the nodes represent the domain's variables (in a medical diagnostic application t... |

38 |
Comparing the calibration and coherence numerical and verbal probability judgments
- Wallsten, Budescu, et al.
- 1993
(Show Context)
Citation Context ...and they have a fixed rank-order. A hundred percent is always more than seventy-five percent. Words are, in comparison, vaguer, they do not allow calculations and they are more variably interpretable =-=[54]. This dis-=-advantage of words is visible in the quite consistent finding that the interpretation of verbally expressed probabilities is contextdependent [5,56]. If winning a lottery is "possible", ente... |

36 | Why is diagnosis using belief networks insensitive to imprecision in probabilities
- Henrion, Pradhan, et al.
- 1996
(Show Context)
Citation Context ...these probabilities numerically, something they are often reluctant to do. They do not feel familiar enough with the concept of probability or they find it di#cult to attach a number to their beliefs =-=[23]-=-. On the output side, explanations of the results of computations of the network in terms of variables with numerical probabilities may be uncomfortable. Researchers have recognised the importance of ... |

35 | How to elicit many probabilities
- Gaag, Renooij, et al.
- 1999
(Show Context)
Citation Context ...ssments numerically or to mark them on a horizontal line, were quite unsuccessful. Now, with our double scale, elicitation proceeds much more e#ectively, to a much greater satisfaction of the experts =-=[51]-=-. Some probabilities they easily give as a number, for others they use the verbal expressions and then check the scale at or near the expression that best fits their estimate. We will continue this st... |

31 | Elicitation of Probabilities for Belief Networks: Combining Qualitative and Quantitative Information - Druzdzel, Gaag - 1995 |

28 | Representing uncertain knowledge: an artificial intelligence approach - Krause, Clark - 1993 |

27 | Qualitative verbal explanations in bayesian belief networks
- Druzdzel
- 1996
(Show Context)
Citation Context ... providing users with more easily understandable explanations of the results, for which numbers may not necessarily be the best option and verbal probability expressions are seen as good alternatives =-=[16,13]-=-. Except in situations where the odds are objectively measurable, most people feel more at ease with verbal probability expressions than with numbers. When people communicate probabilities, they frequ... |

27 |
Quantifying probabilistic expressions
- Mosteller, Youtz
- 1990
(Show Context)
Citation Context ...ty is generally not judged to be wise. Moreover, personal opinions about the consequences of the events referred to result in individual variations in the meanings assigned to probability expressions =-=[34]-=-. Some people may not mind meeting a disliked person or even enjoy the confrontation, others may definitely wish to avoid it. Studies of numerical versus verbal probabilities generally ask subjects to... |

26 |
Computerbased decision-support in the management of primary gastric non-Hodgkin lymphoma. Meth Inform Med
- PJF, Boot, et al.
- 1998
(Show Context)
Citation Context ...orks in di#erent problem domains have been developed, which demonstrates that they have established their position in Artificial Intelligence as valuable representations of reasoning with uncertainty =-=[1,25,28,12,31]-=- . A belief network consists of a qualitative and a quantitative part. The qualitative part is a directed graph, where the nodes represent the domain's variables (in a medical diagnostic application t... |

25 |
How probable is probable? A numerical translation of verbal probability expressions
- Beyth-Marom
- 1982
(Show Context)
Citation Context ... vice versa. In such studies, researchers have found a great between subject variability in the numerical values assigned to verbal probabilities and great overlap between the words (see among others =-=[2,53]-=-). Much less within and between subject variability was found in the numerical probability expressions subjects used when asked to describe a graphical representation of chance than in the verbal expr... |

22 | Decisions based on numerically and verbally expressed uncertainties,Journal of Experimental Psychology: Human Perception and Performance - Budescu, Weinberg, et al. - 1988 |

22 |
Representing uncertain knowledge
- Krause, Clark
- 1993
(Show Context)
Citation Context ...hat is intended to reason with probabilities and to communicate the results of that reasoning to users, the mode in which people normally represent probability needs to be taken into account (compare =-=[29,52]-=-). However, it is not obvious which verbal probability expressions could then be used. We therefore investigate whether we can find a set of verbal probability expressionswhose meaning is agreed upon ... |

21 |
Verbal probabilities: ambiguous, context-dependent, or both
- Brun, Teigen
- 1988
(Show Context)
Citation Context ...ns and they are more variably interpretable [54]. This disadvantage of words is visible in the quite consistent finding that the interpretation of verbally expressed probabilities is contextdependent =-=[5,56]. If winning a lotte-=-ry is "possible", entering the lottery may generally be seen as a good thing to do, while if encountering a much disliked person at a party is "possible", going to that party is ge... |

20 |
Preference and reasons for communicating probabilistic information in numerical or verbal terms
- Wallsten, Budescu, et al.
- 1993
(Show Context)
Citation Context ...one person in three prefers numbers for both expressing and receiving information, the second prefers words for both, and the third indeed betrays Erev & Cohen's communication mode preference paradox =-=[55]-=-. But this preference was not firm, neither for Erev and Cohen's subjects nor for the subjects in a study by Rapoport and colleagues [41]. Subjects were found to be willing and able to use both modes ... |

18 |
Consistency in interpretation of probabilistic phrases
- Budescu, Wallsten
- 1985
(Show Context)
Citation Context ...epresentation of chance than in the verbal expressions they used [8]. Subjects were found to be consistent with themselves in their interpretations of verbal expressions, but much less so with others =-=[7]-=-. Physicians are no exceptions. When they were asked to give the meanings of verbal probability expressions by marking them on a 0 -- 100 percentage scale [15] or when they were asked to translate ver... |

14 | Cohen: Verbal versus numerical probabilities: efficiency, biases, and the preference paradox - Erev, B |

13 |
Patterns of preference for numerical and verbal probabilities
- Olsen, Budescu
- 1997
(Show Context)
Citation Context ... result in uniformly better processing than the other. The conclusion seems to be that there are no grounds to prefer either numerical or verbal probability expressions as the better medium (cf. also =-=[8,17,21,46,38]-=-). It has also been suggested that in some situations clearly interpretable verbal expressions are preferable to numbers. For example, Wallsten et al. [55] advise probability judgements to be elicited... |

12 | Contextual effects in the interpretations of probability words: Perceived base rate and severity of events - Weber, Hilton - 1990 |

10 |
How medical professionals evaluate expressions of probability
- Kong, Bamett, et al.
- 1986
(Show Context)
Citation Context ...ound in the rank ordering of verbal probability expressions by general practitioners [37], and individuals were found to have a relatively stable rank ordering of verbal probability phrases over time =-=[7,27]-=-. Our suggestion is that when probabilities are elicited from experts, for example when constructing a belief network, the experts be shown a scale, depicted graphically as a vertical line with number... |

10 |
Numeric and linguistic information representation in multiattribute choice
- Stone, Schkade
- 1991
(Show Context)
Citation Context ... result in uniformly better processing than the other. The conclusion seems to be that there are no grounds to prefer either numerical or verbal probability expressions as the better medium (cf. also =-=[8,17,21,46,38]-=-). It has also been suggested that in some situations clearly interpretable verbal expressions are preferable to numbers. For example, Wallsten et al. [55] advise probability judgements to be elicited... |

9 |
Expression of probability: words and numbers
- Bryant
- 1980
(Show Context)
Citation Context ...re asked to give the meanings of verbal probability expressions by marking them on a 0 -- 100 percentage scale [15] or when they were asked to translate verbal expressions into numerical expressions (=-=[6]-=-, see also [43]) physicians regularly gave di#erent interpretations. When probability information was communicated by verbal expressions, interpretations were also found to be highly imprecise, presum... |

9 | An applied study using the analytic hierarchy process to translate common verbal phrases to numerical probabilities
- Tavana, Kennedy, et al.
- 1997
(Show Context)
Citation Context ...e di#erent subjects. 11 Table 4 Mean ranks (and standard deviations) of the eight probability expressions by the subjects in the no context condition (groups 1 and 2) and as reported by Tavana et al. =-=[48]-=-, by Budescu & Wallsten [7]and by Clark [9], studies 5.4 and 5.2 no context Tavana Budescu Clark (this study) study 5.4 study 5.2 (n = 48) (n = 30) (n = 32) (n = 16) (n = 16) certain 1.15 (0.94) 1.05 ... |

8 |
a Bayesian expert system for echocardiography
- Dõez, Mira, et al.
(Show Context)
Citation Context ...orks in di#erent problem domains have been developed, which demonstrates that they have established their position in Artificial Intelligence as valuable representations of reasoning with uncertainty =-=[1,25,28,12,31]-=- . A belief network consists of a qualitative and a quantitative part. The qualitative part is a directed graph, where the nodes represent the domain's variables (in a medical diagnostic application t... |

7 |
Explanation of probabilistic inference
- Elsaesser
- 1989
(Show Context)
Citation Context ... providing users with more easily understandable explanations of the results, for which numbers may not necessarily be the best option and verbal probability expressions are seen as good alternatives =-=[16,13]-=-. Except in situations where the odds are objectively measurable, most people feel more at ease with verbal probability expressions than with numbers. When people communicate probabilities, they frequ... |

7 |
Use of Causal Probabilistic Networks as High Level Models in Computer Vision
- Jensen, Nielsen, et al.
- 1990
(Show Context)
Citation Context |

6 |
Selection of verbal probabilities: A solution for some problems of verbal probability expression
- Hamm
- 1991
(Show Context)
Citation Context ...d and could be used [26]. A small number of carefully selected expressions seems best [7,42], or a table could be presented codifying the numerical meaning of the verbal phrases (compare among others =-=[22,18]-=-). Physicians could then continue to use verbal expressions if they prefer, but with more consistency of terminology [32]. A scale with a small number of expressions is recommended because it would be... |

6 | Critical decisions under uncertainty: representa tion and structure. Copi tive Science
- Kuipers, Moskowitr, et al.
- 1988
(Show Context)
Citation Context ... in communicating probabilities to their patients. Other researchers report similar results. Physicians and other health workers express and process probabilities in verbal rather than numerical form =-=[32,30]-=-. Physicians rarely reason using numerical probabilities, and if they do, they tend to make errors [14,49]. Kuipers and colleagues [30] conclude that to physicians, subjective probabilities are not nu... |

6 | Verbal expressions of probability in informed consent litigation
- Merz, Druzdzel, et al.
- 1991
(Show Context)
Citation Context ...communicated by verbal expressions, interpretations were also found to be highly imprecise, presumably because they were influenced by the severity of the consequences associated with the information =-=[32]. For example, "-=-;low probability of infection" was interpreted di#erently than "low risk of death". Most of the authors referred to above conclude that physicians should use numerical, not verbal, expr... |

6 |
Nonmetric individual di erences multidimensional scaling: an alternating least squares method with optimal scaling features
- Takane, Young, et al.
- 1977
(Show Context)
Citation Context ...et of distances between objects and creates the 'map' by computing the positions (co-ordinates) of the objects. We used the SPSS module ALSCAL (Alternating Least-Squares Scaling) as our MDS procedure =-=[47]-=-. The type of MDS we used was Replicated MDS, which computes a single MDS solution for all matrices together and uses the Euclidean distance model (an n-dimensional version of the Pythagorean theorem)... |

5 |
Alternatives to Bayes? A quantitative comparison with rule-based diagnostic inference
- Fox, Barber, et al.
- 1980
(Show Context)
Citation Context ...c. On the input side, it involves the elicitation from domain experts of (conditional) probabilities for all variables. This may be a prohibitive quantity of probabilities, even in restricted domains =-=[19]-=-. What's more, experts are required to express all these probabilities numerically, something they are often reluctant to do. They do not feel familiar enough with the concept of probability or they f... |

4 |
Numbers are better than words. Verbal specifications of frequency have no place in medicine
- MA, Axelrod
- 1983
(Show Context)
Citation Context ...f infection" was interpreted di#erently than "low risk of death". Most of the authors referred to above conclude that physicians should use numerical, not verbal, expressions of probabi=-=lity (see also [35]-=-). Verbal probability expressions may lead to confusion, therefore numbers should be used [7]. 3 However, ceteris paribus, verbal expressions of probability are perceived as more natural than numerica... |

4 |
A sensitivity analysis of pathfinder: A follow-up study
- Ng, Abramson
- 1991
(Show Context)
Citation Context ...uggested that, even if verbal expressions would be less precise than numerical expressions, imprecision of probabilities does not entail a deterioration in the average performance of a belief network =-=[23,40,36]-=-. This suggestion awaits further corroboration. Limiting the number of verbal expressions in a scale might, however, be advisable. The use of some verbal expressions should be avoided, to wit those fo... |

4 |
Revision of opinion with verbally and numerically expressed uncertainties Acta Psychologica
- Rapoport, Wallsten, et al.
- 1990
(Show Context)
Citation Context ...d betrays Erev & Cohen's communication mode preference paradox [55]. But this preference was not firm, neither for Erev and Cohen's subjects nor for the subjects in a study by Rapoport and colleagues =-=[41]-=-. Subjects were found to be willing and able to use both modes of expression. Brun and Teigen [5] found that physicians preferred to use words in communicating probabilities to their patients. Other r... |

3 |
A causal probabilistic network for interpretation of electromyographic findings
- MUNIN
- 1987
(Show Context)
Citation Context |

3 |
Rarely occurring headaches and rarely occurring blindness: Is rarely 5 rarely? The meaning of verbal frequentistic labels in specific medical contexts
- Fischer, Jungermann
- 1996
(Show Context)
Citation Context ...d and could be used [26]. A small number of carefully selected expressions seems best [7,42], or a table could be presented codifying the numerical meaning of the verbal phrases (compare among others =-=[22,18]-=-). Physicians could then continue to use verbal expressions if they prefer, but with more consistency of terminology [32]. A scale with a small number of expressions is recommended because it would be... |

3 |
Introduction to Multidimensional Scaling : Theory
- Schiman
- 1981
(Show Context)
Citation Context ...riment we asked subjects for pairwise comparison, that is: for similarity judgements among all pairs of verbal probability expressions, to uncover the underlying structure of relationships among them =-=[44,4]-=-. For the eight expressions, there were 28 pairs to compare. A similarity judgement was made by scoring each pair of expressions on a 10 cm. line, using as anchors the expressions 'exact same' and 'co... |

3 |
Verbal versus numerical processing of subjective probabilities
- Zimmer
- 1983
(Show Context)
Citation Context ...t and 14 in a shorter version [5], as few as two [21] or as many as 52 [34]. These lists were compiled by the researchers, which does not guarantee that people would actually use them. Indeed, Zimmer =-=[57]-=- found that when subjects were asked directly for verbal descriptions of probability, the mean number of expressions used was 5.44. Di#erences in interpretation of the verbal expressions may be preven... |

3 | Explanation of Probabilistic Inference,” Uncertainty - Elsaesser - 1989 |

2 |
Determinants of numerical versus verbal probabilities
- Gonzalez, Frenck-Mestre
- 1993
(Show Context)
Citation Context ... result in uniformly better processing than the other. The conclusion seems to be that there are no grounds to prefer either numerical or verbal probability expressions as the better medium (cf. also =-=[8,17,21,46,38]-=-). It has also been suggested that in some situations clearly interpretable verbal expressions are preferable to numbers. For example, Wallsten et al. [55] advise probability judgements to be elicited... |