Reevaluating Amdahl's Law

Reevaluating Amdahl's Law (1988)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by John L. Gustafson

Venue:	Communications of the ACM
Citations:	317 - 4 self

BibTeX

@ARTICLE{Gustafson88reevaluatingamdahl's,
    author = {John L. Gustafson},
    title = {Reevaluating Amdahl's Law},
    journal = {Communications of the ACM},
    year = {1988},
    volume = {31},
    pages = {532--533}
}

OpenURL

Abstract

At Sandia National Laboratories, we are currently engaged in research involving massively-parallel processing. There is considerable skepticism regarding the viability of massive parallelism; the skepticism centers around Amdahl’s law, an argument put forth by Gene Amdahl in 1967 [1] that even when the fraction of serial work in a given problem is small, say s, the maximum speedup obtainable from even an infinite number of parallel processors is only 1/s. We now have timing results for a 1024-processor system that demonstrate that the assumptions underlying Amdahl’s 1967 argument are inappropriate for the current approach to massive ensemble parallelism. If N is the number of processors, s is the amount of time spent (by a serial processor) on serial parts of a program and p is the amount of time spent (by a serial processor) on parts of the program that can be done in parallel, then Amdahl’s law says that speedup is given by Speedup = (s + p) ⁄ ( s + p ⁄ N) = 1 ⁄ ( s + p ⁄ N), where we have set total time s␣+␣p ␣=␣1 for algebraic simplicity. For N = 1024, this is an unforgivingly steep function of s near s = 0 (see Figure 1). The steepness of the graph near s = 0 (approximately – N 2) implies that very few problems will experience even a 100-fold speedup. Yet for three very practical applications (s = 0.4 – 0.8 percent) used at Sandia, we have achieved the speedup factors on a 1024-processor hypercube which we believe are unprecedented [2]: 1021 for beam stress analysis using conjugate gradients, 1020 for baffled surface wave simulation using explicit finite differences, and 1016 for unstable fluid flow using flux-corrected transport. How can this be, when Amdahl’s argument would predict otherwise?

Keyphrases

serial processor amdahl law skepticism center serial work infinite number current approach massively-parallel processing algebraic simplicity maximum speedup obtainable surface wave simulation amdahl argument gene amdahl parallel processor explicit finite difference 100-fold speedup practical application beam stress analysis conjugate gradient 1024-processor hypercube flux-corrected transport massive parallelism unstable fluid flow serial part steep function ensemble parallelism total time 1024-processor system speedup factor sandia national laboratory considerable skepticism