Abstract. We present a semi-formal analysis method for fault-tolerant distributed algorithms written in the distributed functional programming language Erlang. In this setting, standard model checking techniques are often too expensive or too limiting, whereas testing techniques often do not cover enough of the state space. Our idea is to first run instances of the algorithm on generated stimuli, thereby creating traces of events and states. Then, using an abstraction function specified by the user, our tool generates from these traces an abstract state transition diagram of the system, which can be nicely visualized and thus greatly helps in debugging the system. Lastly, formal requirements of the system specified in temporal logic can be checked automatically to hold for the generated abstract state transition diagram. Because the state transition diagram is abstract, we know that the checked requirements hold for a lot more traces than just the traces we actually ran. We have applied our method to a commonly used open-source fault-tolerant leader election algorithm, and discovered two serious bugs. We have also implemented a new algorithm that does not have these bugs. 1

It is now ten years ago since the decision was made to apply the functional programming language Erlang to real production projects at Ericsson. In late 1995, development on the Open Telecom Platform (OTP) started, and in mid 1996 the AXD 301 project became the first user of OTP. The AXD 301 Multi-service Switch was released in October 1998, and later became "the heart of ENGINE", Ericsson's leading Voice over Packet solution.

We propose an extension to Fredlund’s formal semantics for Erlang that models the concept of nodes. The motivation is that there exist sequences of events that can occur in practice, but are impossible to describe using a single-node semantics, such as Fredlund’s. The consequence is that some errors in distributed systems might not be detected by model checkers based on Fredlund’s original semantics, or by other single-node verification techniques such as testing. Our extension is modest; it re-uses most of Fredlund’s work but adds an extra layer at the top-level.

We present a model checker for verifying distributed programs written in the Erlang programming language. Providing a model checker for Erlang is especially rewarding since the language is by now being seen as a very capable platform for developing industrial strength distributed applications with excellent failure tolerance characteristics. In contrast to most other Erlang verification attempts, we provide support for a very substantial part of the language. The model checker has full Erlang data type support, support for general process communication, node semantics (inter-process behave subtly different from intra-process communication), fault detection and fault tolerance through process linking, and can verify programs written using the OTP Erlang component library (used by most modern Erlang programs). As the model checking tool is itself implemented in Erlang we benefit from the advantages that a (dynamically typed) functional programming language offers: easy prototyping and experimentation with new verification algorithms, rich executable models that use complex data structures directly programmed in Erlang, the ability to treat executable models interchangeably as programs (to be executed directly by the Erlang interpreter) and data, and not least the possibility to cleanly structure and to cleanly combine various verification sub-tasks. In the paper we discuss the design of the tool and provide early indications on its performance.

Abstract—The actor programming model offers a promising model for developing reliable parallel and distributed code. Actors provide flexibility and scalability: local execution may be interleaved, and distributed nodes may operate asynchronously. The resulting nondeterminism is captured by nondeterministic processing of messages. To automate testing, researchers have developed several tools tailored to specific actor systems. As actor languages and libraries continue to evolve, such tools have to be reimplemented. Because many actor systems are compiled to Java bytecode, we have developed Basset, a general framework for testing actor systems compiled to Java bytecode. We illustrate Basset by instantiating it for the Scala programming language and for the ActorFoundry library for Java. Our implementation builds on Java PathFinder, a widely used model checker for Java. Experiments show that Basset can effectively explore executions of actor programs; e.g., it discovered a previously unknown bug in a Scala application. I.

We present the new model checker McErlang for verifying Erlang programs. In comparison with the etomcrl tool set, McErlang differs mainly in that it is implemented in Erlang. The implementation language offers several advantages: checkable programs use “almost” normal Erlang, correctness properties are formulated in Erlang itself instead of a temporal logic, and it is easier to properly diagnose program bugs discovered by the model checker. In addition the model checker can easily be modified, thanks largely to the use of Erlang. The drawback of writing the model checker in Erlang is, potentially, severely reduced performance compared with model checking tools programmed in programming languages which permit destructive updates of data structures.

Abstract. Actor programs consist of a number of concurrent objects called actors, which communicate by exchanging messages. Nondeterminism in actors results from the different possible orders in which available messages are processed. Systematic testing of actor programs explores various feasible message processing schedules. Dynamic partial-order reduction (DPOR) techniques speed up systematic testing by pruning parts of the exploration space. Based on the exploration of a schedule, a DPOR algorithm may find that it need not explore some other schedules. However, the potential pruning that can be achieved using DPOR is highly dependent on the order in which messages are considered for processing. This paper evaluates a number of heuristics for choosing the order in which messages are explored for actor programs, and summarizes their advantages and disadvantages. 1

In order to formally reason about distributed Erlang systems, it is necessary to have a formal semantics. In a previous paper we have proposed such a semantics for distributed Erlang. However, recent work with a model checker for Erlang revealed that the previous attempt was not good enough. In this paper we present a more accurate semantics for distributed Erlang. The more accurate semantics includes several modifications and additions to the semantics for distributed Erlang proposed by Claessen and Svensson in 2005, which in turn is an extension to Fredlund’s formal single-node semantics for Erlang. The most distinct addition to the previous semantics is the possibility to correctly model disconnected nodes.

Erlang is a concurrent functional language, especially tailored for distributed and fault-tolerant software. Its strength has been demonstrated by several successful commercial applications. An important part of Erlang is its support for fault tolerance by implementing failure-recovery, through organising the processes of an Erlang system into trees of processes, in which parent nodes monitor the failure status of their children and are responsible for their restart. In this paper we present ongoing work on a tool that captures the static part of the global process structure of an application Erlang. The tool extracts the process structure from the applications source code, and presents it as a process creation graph. The extracted process structure provides a starting point for understanding and analysis of aspects such as fault handling in Erlang applications.

Algorithms described in literature can often be used to solve practical, industrial problems. In safety-critical industrial settings, algorithms that have been formally verified should be even more attractive candidates for implementations. Nevertheless, we observe little transfer of algorithms from research papers into products. In this paper we describe a case study on the implementation of algorithms for the widely known and broadly studied problem of leader election. Despite thousands of articles on that topic, it still requires a lot of engineering to select the relevant articles, and get a correct algorithm implemented in an industrial setting. Modifications are necessary to meet all requirements. We propose adaptation and testing of formal properties as a realistic and cheap way to check the correctness of the modifications, since performing a formal proof seems unrealistic for industrial systems. We show how we use the properties stated in the articles to guide our tests.