## Random Number Generators for Parallel Computers (1997)

Venue: | The NHSE Review |

Citations: | 25 - 1 self |

### BibTeX

@ARTICLE{Coddington97randomnumber,

author = {Paul Coddington},

title = {Random Number Generators for Parallel Computers},

journal = {The NHSE Review},

year = {1997},

volume = {2}

}

### OpenURL

### Abstract

Random number generators are used in many applications, from slot machines to simulations of nuclear reactors. For many computational science applications, such as Monte Carlo simulation, it is crucial that the generators have good randomness properties. This is particularly true for large-scale simulations done on high-performance parallel computers. Good random number generators are hard to find, and many widely-used techniques have been shown to be inadequate. Finding high-quality, efficient algorithms for random number generation on parallel computers is even more difficult. Here we present a review of the most commonly-used random number generators for parallel computers, and evaluate each generator based on theoretical knowledge and empirical tests. In conclusion, we provide recommendations for using random number generators on parallel computers. Outline This review is organized as follows: A brief summary of the findings of this review is first presented, giving an overview of the use of parallel random number generators and a list of recommended algorithms. Section 1 is an introduction to random number generators and their use in computer simulations on parallel computers. Section 2 is a summary of the methods used to test and evaluate random number generators, on both sequential and parallel computers. Section 3 gives an overview of the main algorithms used to implement random number generators on sequential computers, provides examples of software implementations of the algorithms, and states any known problems with the algorithms or implementations. Section 4 gives a description of the most common methods used to parallelize the sequential algorithms, provides examples of software implementing these algorithms, and states any known problems ...

### Citations

393 |
M.: The High Performance Fortran Handbook
- Koelbel, Loveman, et al.
- 1994
(Show Context)
Citation Context ...mbers of processors, and for the special case of a single processor. This is beneficial for debugging purposes, and is a requirement for some parallel languages such as High Performance Fortran (HPF) =-=[53]-=- (which provides a random number generator as an intrinsic function, as does Fortran 90). 5. The algorithm should be efficient, which in practice means there should be no data movement between process... |

240 |
Shift Register Sequences
- Golomb
- 1967
(Show Context)
Citation Context ...tic. Many Fortran compilers allow the calling of subroutines written in C. The randomness properties of LFGs can be improved (without sacrificing too much in speed) by using multiple lags (or "ta=-=ps") [39, 28, 22, 14, 15, 17]-=-, i.e. by combining three or more previous elements of the sequence, rather than two. This type of generator has not yet been widely used or studied, however it seems likely that a 3- or 4-lag additiv... |

231 | Random Number Generators: Good Ones Are Hard To Find - Park, Miller - 1988 |

105 |
E¢ cient and portable combined random number generators
- L’Ecuyer
- 1988
(Show Context)
Citation Context ...ts of the numbers produced are correlated, and a "scatter-plot" of ordered tuples (x i ; x i+1 , . . . ) of random floating point numbers plotted in the unit hypercube shows regular lattice =-=structure [2, 27, 32, 33, 7]-=-. This problem becomes worse in higher dimensions, which may affect some high-dimensional simulations. The problem of lattice structure can be quantified using the Spectral Test [1]. Examples of LCGs ... |

87 |
A current view of random number generators
- Marsaglia
- 1985
(Show Context)
Citation Context ...ts of the numbers produced are correlated, and a "scatter-plot" of ordered tuples (x i ; x i+1 , . . . ) of random floating point numbers plotted in the unit hypercube shows regular lattice =-=structure [2, 27, 32, 33, 7]-=-. This problem becomes worse in higher dimensions, which may affect some high-dimensional simulations. The problem of lattice structure can be quantified using the Spectral Test [1]. Examples of LCGs ... |

74 |
Random sequence generation by cellular automata
- Wolfram
- 1986
(Show Context)
Citation Context ...st methods currently available for generating random numbers on parallel computers, as long as the initialization of the lag tables is done properly. 4.4 Other Methods The cellular automata generator =-=[58]-=- is a generalization of the shift register generator, based on cellular automata rules. A parallel version called CMF RANDOM is provided by Thinking Machines [59]. Both the sequential and parallel ver... |

73 |
Random numbers fall mainly in the planes
- Marsaglia
- 1968
(Show Context)
Citation Context ...ts of the numbers produced are correlated, and a "scatter-plot" of ordered tuples (x i ; x i+1 , . . . ) of random floating point numbers plotted in the unit hypercube shows regular lattice =-=structure [2, 27, 32, 33, 7]-=-. This problem becomes worse in higher dimensions, which may affect some high-dimensional simulations. The problem of lattice structure can be quantified using the Spectral Test [1]. Examples of LCGs ... |

60 | A review of pseudorandom number generators - James - 1990 |

60 |
Toward a Universal Random Number Generator
- Marsaglia, Zaman
- 1987
(Show Context)
Citation Context ...G. Other proposed combined generators include algorithms combining an LFG with an LCG [2], or an LFG with a simple Weyl (or arithmetic sequence) generator, which is the basis for the RANMAR generator =-=[5, 45]-=- commonly used in computational physics applications. The addition of the Weyl generator greatly improves the randomness properties over the single additive LFG, but RANMAR still fails some Monte Carl... |

57 |
Random number generators on vector supercomputers and other advanced architecture
- Anderson
- 1990
(Show Context)
Citation Context ...simulations on parallel supercomputers, which consume huge quantities of random numbers, and require parallel algorithms for random number generation. As noted in a number of previous review articles =-=[1, 2, 3, 4, 5, 6, 7]-=-, random number generators provided by computer vendors or recommended in some papers and computer science texts have often been of poor quality. Even generators that perform well in standard statisti... |

56 |
Mathematical Methods in Large-Scale Computing Units
- Lehmer
- 1951
(Show Context)
Citation Context ...found in a number of other review articles [1, 2, 3, 4, 5, 6, 7]. 3.1 Linear Congruential Generators Probably the most commonly-used random number generators are linear congruential generators (LCGs) =-=[37, 1, 3, 5]-=-. The standard C and Unix generators RAND (32-bit precision), DRAND48 and RANF (48-bit precision) are of this type. LCGs produce a sequence X i of random integers using the relation X i = (asX i\Gamma... |

53 |
Random numbers for simulation
- L’Ecuyer
- 1990
(Show Context)
Citation Context ...simulations on parallel supercomputers, which consume huge quantities of random numbers, and require parallel algorithms for random number generation. As noted in a number of previous review articles =-=[1, 2, 3, 4, 5, 6, 7]-=-, random number generators provided by computer vendors or recommended in some papers and computer science texts have often been of poor quality. Even generators that perform well in standard statisti... |

50 |
Monte Carlo simulations: hidden errors from "good" random number generators
- Ferrenberg, Landau, et al.
- 1992
(Show Context)
Citation Context ...f poor quality. Even generators that perform well in standard statistical tests for randomness have sometimes proven to be unreliable for certain applications, particularly in Monte Carlo simulations =-=[8, 9, 10, 11, 12, 13, 14, 15, 16, 17]-=-. The many problems caused in the past by inadequate random number generators on sequential and vector computers are likely to be repeated in a new generation of simulations using parallel computers, ... |

48 |
Random Numbers Generated by Linear Recurrence Modulo-Two
- Tausworthe
- 1965
(Show Context)
Citation Context ...studied, however it seems likely that a 3- or 4-lag additive LFG would be a very fast and effective random number generator. 10 3.3 Shift Register Generators Shift register (or Tausworthe) generators =-=[1, 2, 38, 21, 39]-=- are generally used in a form where they can be considered as a special case of a lagged Fibonacci generator using XOR. XOR gives by far the worst randomness properties of any operation for an LFG [1,... |

43 |
Implementing a Random Number Package with Splitting Facilities
- L’Ecuyer, Cote
- 1991
(Show Context)
Citation Context ... multiplier a and the additive constant c by new values A = a N and C = c(a N \Gamma 1)=(a \Gamma 1) (both modulo M) [47, 48, 49, 50]. Jumping ahead in the sequence can also be done for combined LCGs =-=[6, 51]-=- and shift-register generators [52], but is not practical for LFGs using addition or multiplication, since the computations are much more complex, making it too slow for practical use. A 48-bit LCG us... |

42 |
An efficient and portable pseudo-random number generator
- Wichman, Hill
- 1982
(Show Context)
Citation Context ...ent generators has been shown (both theoretically and empirically) to produce an improved quality generator in many circumstances [2, 32, 6, 42]. Based on an algorithm introduced by Wichmann and Hill =-=[43]-=-, L'Ecuyer [32] has shown how to additively combine two different 32-bit LCGs to produce a generator that passes all known statistical tests and has a long period of around 10 18 , thus overcoming the... |

40 |
A very fast shift-register sequence random number generator
- Kirkpatrick, Stoll
- 1981
(Show Context)
Citation Context ...h known exact results that would occur if the numbers were truly random. Tests of this kind include Monte Carlo simulation [19, 20] of exactly solvable systems such as the two dimensional Ising model =-=[10, 21, 13, 15, 16, 17]-=-, simulations of percolation models [22], and random walks [14, 22, 17]. Generators that pass standard statistical tests have sometimes been found to fail these application-specific tests. It is there... |

39 |
On primitive trinomials (mod 2
- Zierler, Brillhart
- 1968
(Show Context)
Citation Context ...2 p \Gamma 1 for XOR, (2 p \Gamma 1)2 b\Gamma1 for addition and subtraction, and (2 p \Gamma 1)2 b\Gamma3 for multiplication [28, 7, 1, 2, 31]. Tables of suitable lags are available in the literature =-=[1, 28, 30, 29, 31]-=-. An advantage of this generator is that the period can be made arbitrarily large by just increasing the lag p. This also improves the randomness properties [2, 15], since smaller lags mean higher cor... |

34 |
Parallelization of random number generators and long-range correlations, Numerische Mathematik 53
- Matteis, Pagnutti
- 1988
(Show Context)
Citation Context ... fast and convenient to implement this on a computer. However this approach produces highly correlated low-order bits [1, 4, 7], as well as long-range correlations for intervals that are a power of 2 =-=[8, 11, 34, 35, 36]-=-. This can cause problems for certain types of simulations, for example when using a hypercubic grid with a size that is a power of 2, or for applications that expect the lower-order bits to be random... |

30 |
Matrices and the structure of random number sequences. Linear Algebra and its Applications 67
- Marsaglia, Tsay
- 1985
(Show Context)
Citation Context ...he exponents of a primitive polynomial [1, 28]. In that case the period is 2 p \Gamma 1 for XOR, (2 p \Gamma 1)2 b\Gamma1 for addition and subtraction, and (2 p \Gamma 1)2 b\Gamma3 for multiplication =-=[28, 7, 1, 2, 31]-=-. Tables of suitable lags are available in the literature [1, 28, 30, 29, 31]. An advantage of this generator is that the period can be made arbitrarily large by just increasing the lag p. This also i... |

28 | On the Periods of Generalized Fibonacci Recurrences
- Brent
- 1994
(Show Context)
Citation Context ...domness properties and the largest possible period. If M is taken to be 2 b (i.e. the X's have b-bit precision), the maximal period is obtained if the lags are the exponents of a primitive polynomial =-=[1, 28]-=-. In that case the period is 2 p \Gamma 1 for XOR, (2 p \Gamma 1)2 b\Gamma1 for addition and subtraction, and (2 p \Gamma 1)2 b\Gamma3 for multiplication [28, 7, 1, 2, 31]. Tables of suitable lags are... |

28 |
Multiplicative congruential random number generators with modulus 2 β : An exhaustive analysis for β = 32 and a partial analysis for β = 48
- Fishman
- 1990
(Show Context)
Citation Context ...imensional simulations. The problem of lattice structure can be quantified using the Spectral Test [1]. Examples of LCGs with good parameters that perform well in the Spectral Test are given in Refs. =-=[1, 40, 15, 41]-=-. Another problem is that many commonly-used LCGs (including DRAND48 and RANF) use a modulus M that is a power of 2, since it is fast and convenient to implement this on a computer. However this appro... |

26 | Uniform random number generators for supercomputers
- Brent
- 1992
(Show Context)
Citation Context ...me modulus. 4.2 Sequence Splitting Another method for parallelizing random number generators is to split the sequence into non-overlapping contiguous sections, each generated by a different processor =-=[4, 6, 7]-=-. For example, one could divide the period of the generator by the number of processors, and jump ahead in the sequence by this amount for each processor. Alternatively, the length of each section of ... |

24 |
Structural properties for two classes of combined random number generators, Mathematics of Computation 57
- L’Ecuyer, Tezuka
- 1991
(Show Context)
Citation Context ...ndard 32-bit LCGs. This has been implemented in a program known as RANECU [32, 5]. Combining two LCGs in this way is effectively a more efficient way of implementing an LCG with a much larger modulus =-=[44]-=-. Recently L'Ecuyer et al. [41] have implemented combined 48-bit and 64-bit LCGs and MRGs, with even larger periods and better randomness properties. A combined 32-bit LCG is substantially slower than... |

23 |
Monte Carlo simulations: Hidden errors from "good" random number generators
- Ferrenberg, Landau, et al.
- 1992
(Show Context)
Citation Context ...f poor quality. Even generators that perform well in standard statistical tests for randomness have sometimes proven to be unreliable for certain applications, particularly in Monte Carlo simulations =-=[8, 9, 10, 11, 12, 13, 14, 15, 16, 17]-=-. The many problems caused in the past by inadequate random number generators on sequential and vector computers are likely to be repeated in a new generation of simulations using parallel computers, ... |

23 | Analysis of random number generators using Monte Carlo simulation
- Coddington
- 1994
(Show Context)
Citation Context ...39] are generally used in a form where they can be considered as a special case of a lagged Fibonacci generator using XOR. XOR gives by far the worst randomness properties of any operation for an LFG =-=[1, 2, 15]-=-, so these generators are not recommended. Despite their serious drawbacks, shift register generators have been very popular in the past, mainly because they were comparatively fast. However on modern... |

23 |
An Introduction to Computer Simulation Methods
- Gould, Tobochnik
- 1988
(Show Context)
Citation Context ...rally compare the results obtained using a pseudo-random number generator with known exact results that would occur if the numbers were truly random. Tests of this kind include Monte Carlo simulation =-=[19, 20]-=- of exactly solvable systems such as the two dimensional Ising model [10, 21, 13, 15, 16, 17], simulations of percolation models [22], and random walks [14, 22, 17]. Generators that pass standard stat... |

21 | Physical tests for random numbers in simulations
- Vattulainen, Ala-Nissila, et al.
- 1994
(Show Context)
Citation Context ...f poor quality. Even generators that perform well in standard statistical tests for randomness have sometimes proven to be unreliable for certain applications, particularly in Monte Carlo simulations =-=[8, 9, 10, 11, 12, 13, 14, 15, 16, 17]-=-. The many problems caused in the past by inadequate random number generators on sequential and vector computers are likely to be repeated in a new generation of simulations using parallel computers, ... |

19 |
The Handbook of Random Number Generation and Testing with TESTRAND Computer Code
- Dudewicz, Ralley
- 1981
(Show Context)
Citation Context ...cal tests that statistically compare the results produced by the random number generators with results expected from a truly random sequence of numbers. Many standard tests of this kind are available =-=[1, 2, 18]-=-. In addition to standard statistical tests, it is useful to apply application-specific tests that are more relevant to some of the various applications for which random numbers are used. As with the ... |

19 | Requirements for a Parallel Pseudorandom Number Generator
- Mascagni, Bailey
- 1995
(Show Context)
Citation Context ...h guarantees that each processor will generate a sequence from a different full-period cycle [63]. It has been suggested that this method be established as a standard parallel random number generator =-=[64, 65]-=-, and it has been used for the intrinsic random number generator in the Portland Group PGHPF High Performance Fortran [62]. The method used for seeding the lag tables is similar in complexity to jumpi... |

17 |
Implementation and usage of a portable and reproducible parallel pseudorandom number generator
- Pryor, Cuccaro, et al.
- 1994
(Show Context)
Citation Context ...erators. Any LCG produces a single periodic sequence of numbers, with the different seeds just providing different starting points in the sequence. However, LFGs have many disjoint full-period cycles =-=[31, 63]-=-, so two different seed tables may produce two completely 14 different non-overlapping periodic sequences of numbers. In fact, since the number of such disjoint cycles is 2 (p\Gamma1)(b\Gamma1) for su... |

15 | Techniques for testing the quality of parallel pseudo-random number generators
- Cuccaro, Mascagni, et al.
- 1995
(Show Context)
Citation Context ...owever, new techniques are necessary to test algorithms for generating random numbers on parallel computers, for example to look for correlations between random number streams on different processors =-=[25, 26]-=-. Thus far, very little work has been done in this area. A good empirical test of parallel random number generators is to use them with parallel implementations of the Monte Carlo algorithms used for ... |

11 |
A search for good multiple recursive generators
- L’Ecuyer, Blouin, et al.
- 1993
(Show Context)
Citation Context ...imensional simulations. The problem of lattice structure can be quantified using the Spectral Test [1]. Examples of LCGs with good parameters that perform well in the Spectral Test are given in Refs. =-=[1, 40, 15, 41]-=-. Another problem is that many commonly-used LCGs (including DRAND48 and RANF) use a modulus M that is a power of 2, since it is fast and convenient to implement this on a computer. However this appro... |

11 |
A Random Number Generator for Parallel Computers
- Aluru, Prabhu, et al.
- 1992
(Show Context)
Citation Context ...t c by new values A = a N and C = c(a N \Gamma 1)=(a \Gamma 1) (both modulo M) [47, 48, 49, 50]. Jumping ahead in the sequence can also be done for combined LCGs [6, 51] and shift-register generators =-=[52]-=-, but is not practical for LFGs using addition or multiplication, since the computations are much more complex, making it too slow for practical use. A 48-bit LCG using the leapfrog technique has been... |

10 |
Kankaala,Physical models as tests of randomness, Phys
- Vattulainen, Ala-Nissila, et al.
- 1995
(Show Context)
Citation Context ...tic. Many Fortran compilers allow the calling of subroutines written in C. The randomness properties of LFGs can be improved (without sacrificing too much in speed) by using multiple lags (or "ta=-=ps") [39, 28, 22, 14, 15, 17]-=-, i.e. by combining three or more previous elements of the sequence, rather than two. This type of generator has not yet been widely used or studied, however it seems likely that a 3- or 4-lag additiv... |

9 |
The Art of Computer Programming Vol. 2: Seminumerical Methods (Addison-Wesley
- Knuth
- 1981
(Show Context)
Citation Context ...simulations on parallel supercomputers, which consume huge quantities of random numbers, and require parallel algorithms for random number generation. As noted in a number of previous review articles =-=[1, 2, 3, 4, 5, 6, 7]-=-, random number generators provided by computer vendors or recommended in some papers and computer science texts have often been of poor quality. Even generators that perform well in standard statisti... |

9 | Tests of random number generators using Ising model simulations
- Coddington
- 1996
(Show Context)
Citation Context ... algorithms used for simulating the two dimensional Ising model, which have proven to be very effective at testing sequential generators. Such tests have been done for a number of parallel generators =-=[23, 24]-=-. 3 Random Number Generators In this section we introduce the most common algorithms used in random number generators -- the linear congruential, lagged Fibonacci, and shift register generators, and t... |

9 |
Monte Carlo Tests of Parallel Random Number Generators, in preparation
- Coddington, Rosario, et al.
(Show Context)
Citation Context ... algorithms used for simulating the two dimensional Ising model, which have proven to be very effective at testing sequential generators. Such tests have been done for a number of parallel generators =-=[23, 24]-=-. 3 Random Number Generators In this section we introduce the most common algorithms used in random number generators -- the linear congruential, lagged Fibonacci, and shift register generators, and t... |

8 |
Random number generators for parallel processors
- Eddy
- 1990
(Show Context)
Citation Context ...essors could produce strong inter-processor correlations. In this section we describe some of the most common parallel random number generators. More information is available in other review articles =-=[4, 46]-=- and in the references given in this section. 4.1 The Leapfrog Method Ideally we would like a parallel random number generator to produce the same sequence of random numbers for different numbers of p... |

7 |
Reduction of correlations in shift-register sequence random number generators using multiple feedback taps, unpublished
- Ziff
- 1992
(Show Context)
Citation Context ...ruly random. Tests of this kind include Monte Carlo simulation [19, 20] of exactly solvable systems such as the two dimensional Ising model [10, 21, 13, 15, 16, 17], simulations of percolation models =-=[22]-=-, and random walks [14, 22, 17]. Generators that pass standard statistical tests have sometimes been found to fail these application-specific tests. It is therefore important to use as wide a variety ... |

7 |
A remark on long-range correlations in multiplicative congruential pseudo random number generators
- Eichenauer-Herrmann, Grothe
- 1989
(Show Context)
Citation Context ... fast and convenient to implement this on a computer. However this approach produces highly correlated low-order bits [1, 4, 7], as well as long-range correlations for intervals that are a power of 2 =-=[8, 11, 34, 35, 36]-=-. This can cause problems for certain types of simulations, for example when using a hypercubic grid with a size that is a power of 2, or for applications that expect the lower-order bits to be random... |

6 | Effects of the random number generator on computer simulations”, Phys - Parisi, Rapuano - 1985 |

6 |
Studies of random number generators for parallel processing
- Bowman, Robinson
- 1987
(Show Context)
Citation Context ...is spread across processors in the same way as a deck of cards is dealt in turn to players in a card game. This is known as the leapfrog technique, since each processor leapfrogs by N in the sequence =-=[47, 48, 49, 50, 4]-=-. In order to use this method we need to be able to easily jump ahead in the sequence. This can be done quite easily for linear congruential generators, and merely involves replacing the multiplier a ... |

5 |
K.: Long range correlations in random number generators and their influence on Monte Carlo simulations
- Filk, Marcu, et al.
- 1985
(Show Context)
Citation Context |

5 |
Fluctuations and lack of self-averaging in the kinetics of domain growth
- Milchev, Binder, et al.
- 1986
(Show Context)
Citation Context |

4 |
Problems with the random number generator RANF implemented on
- Kalle, Wansleben
- 1984
(Show Context)
Citation Context |

4 |
Smooth finite-size behavior of the three-dimensional Ising model, Physica 132A
- Hoogland, Compagner, et al.
- 1985
(Show Context)
Citation Context |

4 |
Controlling correlations in parallel Monte
- Matteis, Pagnutti
- 1995
(Show Context)
Citation Context ...owever, new techniques are necessary to test algorithms for generating random numbers on parallel computers, for example to look for correlations between random number streams on different processors =-=[25, 26]-=-. Thus far, very little work has been done in this area. A good empirical test of parallel random number generators is to use them with parallel implementations of the Monte Carlo algorithms used for ... |

4 |
Shift-register sequence random number generators on the hypercube concurrent computers
- Chiu
- 1988
(Show Context)
Citation Context ...e is, however, an even simpler way to parallelize a lagged Fibonacci generator, which is to run the same sequential generator on each processor, but with different initial lag tables (or seed tables) =-=[55, 56]-=-. In fact this technique is no different to what is done on a sequential computer, when a simulation needs to be run many times using different random numbers. In that case, the user just chooses diff... |

3 |
Quasi-Monte Carlo methods and pseudo-random
- Neiderreiter
- 1978
(Show Context)
Citation Context |