We propose a new approach for developing and deploying distributed systems, in which nodes predict distributed consequences of their actions, and use this information to detect and avoid errors. Each node continuously runs a state exploration algorithm on a recent consistent snapshot of its neighborhood and predicts possible future violations of specified safety properties. We describe a new state exploration algorithm, consequence prediction, which explores causally related chains of events that lead to property violation. This paper describes the design and implementation of this approach, termed CrystalBall. We evaluate CrystalBall on RandTree, BulletPrime, Paxos, and Chord distributed system implementations. We identified new bugs in mature Mace implementations of three systems. Furthermore, we show that if the bug is not corrected during system development, CrystalBall is effective in steering the execution away from inconsistent states at runtime.

We propose a novel approach to Pocket Switched Networks (PSNs) [8] using a specialised declarative language called ‘D 3 N’. A PSN is a recently devised type of communication based on physical proximity, where people encounter each other and their devices directly communicate within their communication range. D 3 N allows us to program distributed applications based on reactive behaviour in a distributed set of nodes. We exploit a functional language approach in designing D 3 N for the clean abstraction given by pure declarative languages, at the same time, taking an advantage of well defined semantics. In this paper, we show a fragment of D 3 N, describe the node runtime architecture, and illustrate its effectiveness through some examples.