We describe a new architecture for Byzantine fault tolerant state machine replication that separates agreement that orders requests from execution that processes requests. This separation yields two fundamental and practically significant advantages over previous architectures. First, it reduces replication costs because the new architecture can tolerate faults in up to half of the state machine replicas that execute requests. Previous systems can tolerate faults in at most a third of the combined agreement/state machine replicas. Second, separating agreement from execution allows a general privacy firewall architecture to protect confidentiality through replication. In contrast, replication in previous systems hurts confidentiality because exploiting the weakest replica can be su#cient to compromise the system. We have constructed a prototype and evaluated it running both microbenchmarks and an NFS server. Overall, we find that the architecture adds modest latencies to unreplicated systems and that its performance is competitive with existing Byzantine fault tolerant systems.

This paper affirms that the quantification of life-critical software reliability is infeasible using statistical methods whether applied to standard software or fault-tolerant software. The classical methods of estimating reliability are shown to lead to exhorbitant amounts of testing when applied to life-critical software. Reliability growth models are examined and also shown to be incapable of overcoming the need for excessive amounts of testing. The key assumption of software fault tolerance---separately programmed versions fail independently---is shown to be problematic. This assumption cannot be justified by experimentation in the ultrareliability region and subjective arguments in its favor are not sufficiently strong to justify it as an axiom. Also, the implications of the recent multiversion software experiments support this affirmation. Index Terms---Life-Critical, Validation, Software Reliability, Design Error, Ultrareliability, Software Fault-Tolerance 1 Introducti...

An experiment comparing the effectiveness of the all-uses and all-edges test data adequacy criteria was performed. The experiment was designed so as to overcome some of the deficiencies of previous software testing experiments. A large number of test sets was randomly generated for each of nine subject programs with subtle errors. For each test set, the percentages of executable edges and definition-use associations covered were measured and it was determined whether the test set exposed an error. Hypothesis testing was used to investigate whether all-uses adequate test sets are more likely to expose errors than are all-edges adequate test sets. All-uses was significantly more effective than all-edges for five of the subjects, and appeared guaranteed to detect the error in four of them. Further analysis showed that in four of these subjects, all-uses-adequate test sets were more effective than all-edges-adequate test sets of similar size. Logistic regression analysis was used to investigate whether the probability that a test set exposes an error increases as the percentage of definition-use associations or edges covered by it increases. The evidence did not strongly support this conjecture. Error exposing ability was shown to be strongly positively correlated to percentage of covered definition-use associations in only four of the nine subjects. Error exposing ability was also shown to be positively correlated to the percentage of covered edges in four (different) subjects, but the relationship was weaker. Author's address: Computer Science Dept., Polytechnic University, 6 Metrotech Center, Brooklyn, N.Y. 11201. E-mail: pfrankl@poly.edu. Supported in part by NSF Grants CCR-8810287 and CCR9206910 and by the New York State Science and Technology Founda...

Computer scientists and practitioners defend the lack of experimentation with a wide range of arguments. Some arguments suggest that experimentation may be inappropriate, too difficult, useless, and even harmful. This article discusses several such arguments to illustrate the importance of experimentation for computer science. This is a preprint of an article with the same title that appeared in IEEE Computer, 31(5), May 1998, 32--40. Keywords: Empiricism, experiments, laboratory, scientific method. 1 Is computer science an experimental science? Do computer scientists need to experiment at all? Only if we answer "yes" does it make sense to ask whether there is enough of it. In his Allen Newell Award Lecture, Fred Brooks suggests that computer science is "not a science, but a synthetic, an engineering discipline"[2]. In an engineering field, testing theories by experiments would be misplaced. Brooks and others seem troubled by the fact that the phenomena studied by computer scientists ...

We present Zyzzyva, a protocol that uses speculation to reduce the cost and simplify the design of Byzantine fault tolerant state machine replication. In Zyzzyva, replicas respond to a client’s request without first running an expensive three-phase commit protocol to reach agreement on the order in which the request must be processed. Instead, they optimistically adopt the order proposed by the primary and respond immediately to the client. Replicas can thus become temporarily inconsistent with one another, but clients detect inconsistencies, help correct replicas converge on a single total ordering of requests, and only rely on responses that are consistent with this total order. This approach allows Zyzzyva to reduce replication overheads to near their theoretical minima.

Abstract not found

This paper affirms that quantification of life-critical software reliability is infeasible using statistical methods whether applied to standard software or faulttolerant software. The key assumption of software fault tolerance---separately programmed versions fail independently---is shown to be problematic. This assumption cannot be justified by experimentation in the ultrareliability region and subjective arguments in its favor are not sufficiently strong to justify it as an axiom. Also, the implications of the recent multiversion software experiments support this affirmation. Index Terms: LIFE-CRITICAL, VALIDATION, SOFTWARE RELIABILITY, DESIGN ERROR, ULTRARELIABILITY, SOFTWARE FAULT-TOLERANCE, 1 Introduction The potential of enhanced flexibility and functionality has led to an ever increasing use of digital computer systems in control applications. At first, the digital systems were designed to perform the same functions as their analog counterparts. However, the availability of en...

Abstract-We have conducted a large-scale experiment in N-version programming. A total of 27 versions of a program were prepared in-dependently from the same specification at two universities. The re-sults of executing the versions revealed that the versions were individ-ually extremely reliable but that the number of input cases in which more than one failed was substantially more than would be expected if they were statistically independent. After the versions had been executed, the failures of each version were examined and the associated faults located. In this paper we pre-sent an analysis of these faults. Our goal in undertaking this analysis was to understand better the nature of the faults. We found that in some cases the programmers made equivalent logical errors, indicating that some parts of the problem were simply more difficult than others. We also found cases in which apparently different logical errors yielded faults that caused statistically correlated failures, indicating that there are special cases in the input space that present difficulty in various parts of the solution. A formal model is presented to explain this phe-nomenon. It appears that minor differences in the software develop-ment environment, such as the use of different programming languages for the different versions, would not have a major impact in reducing the incidence of faults that cause correlated failures. Index Terms-Design diversity, fault-tolerant software, multiver-sion programming, N-version programming, software reliability. I.

After a system is deployed, fixes, enhancements, and modifications all occur that change the components that make up the system. Unfortunately, new versions of components can introduce new errors and break existing, depended-upon behavior. When this happens, the old component version could have provided the correct behavior, but it is no longer part of the system. We propose a framework for upgrading system components that, instead of removing the old version of the component, keeps multiple versions of a component running. Doing so allows behavior to be utilized from all versions, and maintains system integrity and correctness even in the presence of newly introduced errors. This framework ensures that the move towards dynamic, configurable software systems does not lessen, but rather provides capabilities to enhance, the reliability that software will achieve through the next century. 1 INTRODUCTION Users fear upgrades. This unfortunate but true statement reflects the current para...

Software design patterns package proven solutions to recurring design problems in a form that simplifies reuse. We are seeking empirical evidence whether using design patterns is beneficial. In particular, one may prefer using a design pattern even if the actual design problem is simpler than that solved by the pattern, i.e., if not all of the functionality offered by the pattern is actually required. Our experiment investigates software maintenance scenarios that employ various design patterns and compares designs with patterns to simpler alternatives. The subjects were professional software engineers. In most of our nine maintenance tasks, we found positive effects from using a design pattern: Either its inherent additional flexibility was achieved without requiring more maintenance time or maintenance time was reduced compared to the simpler alternative. In a few cases, we found negative effects: The alternative solution was less error-prone or required less maintenance time. Although most of these effects were expected, a few were surprising: A negative effect occurs although a certain application of the Observer pattern appears to be well justified and a positive effect occurs despite superfluous flexibility (and, hence, complexity) introduced by a certain application of the Decorator pattern. Overall, we conclude that, unless there is a clear reason to prefer the simpler solution, it is probably wise to choose the flexibility provided by the design pattern because unexpected new requirements often appear. We identify several questions for future empirical research.