Abstract: This paper presents Pheme1, a peer-to-peer data synchronization protocol that can be used to construct new distributed file systems that share data across collections of devices with limited, varying, or intermittent connectivity. The suitability of Pheme for such environments is not matched by existing protocols. With current technology trends, devices will almost-always be connected to a server or to another device, synchronizing different sets of data as they please. Existing protocols often assume a coarse-grained synchronization model which makes changing the data set of an already established synchronization stream expensive. Pheme introduces fine-grained synchronization so that devices can cheaply change the data set they want to synchronize while they are connected to the other device. In addition, it uses a new dependency summary vector (DSV) scheme to detect conflicts with no extra overhead despite network interruptions and exposes flexible commit mechanisms to allow applications to implement their own commit schemes. Because Pheme provides developers with various synchronization options, systems can be built that send the right data via the right network paths and dramatically outperform traditional client-server or server-replication protocols. At the same time, Pheme is more efficient than existing similarly flexible protocols like PRACTI – we observe several orders of magnitude of bandwidth savings in some experiments. 1

My research has focused on constructing robust large-scale distributed systems. The bulk of this work can be understood in the context of two intertwined efforts: constructing cooperative and peer-to-peer services and understanding the fundamental principles of large-scale data replication. Cooperative and peer-to-peer services Cooperative and peer-to-peer services both seek to provide a way to scale services beyond what can be provided by even a high-end server machine. Cooperative services do this by treating a service as a parallel program and then running the program across a cluster of machines. Key challenges include rearchitecting services not only to provide good performance by balancing parallelism and locality but also to ensure good reliability and simple management. Peer-to-peer services go further and enlist machines controlled by different users to collectively provide a service to each other. In addition to the problems of cooperative services, peer-to-peer services have to cope with new issues of trust that arise when a service runs across machines spanning multiple administrative domains with limited trust or competing interests. Some highlights of this stream of work include • Serverless file systems and cooperative caching. We constructed xFS [ADN + 96] to explore an extreme point in the design space of constructing file systems as parallel programs: xFS’s goals included