The LOCKSS project has developed and deployed in a worldwide test a peer-to-peer system for preserving access to journals and other archival information published on the Web. It consists of a large number of independent, low-cost, persistent web caches that cooperate to detect and repair damage to their content by voting in "opinion polls." Based on this experience, we present a design for and simulations of a novel protocol for voting in systems of this kind. It incorporates rate limitation and intrusion detection to ensure that even some very powerful adversaries attacking over many years have only a small probability of causing irrecoverable damage before being detected.

The LOCKSS project has developed and deployed in a worldwide test a peer-to-peer system for preserving access to journals and other archival information published on the Web. It consists of a large number of independent, low-cost, persistent web caches that cooperate to detect and repair damage to their content by voting in "opinion polls." Based on this experience, we present a design for and simulations of a novel protocol for voting in systems of this kind. It incorporates rate limitation and intrusion detection to ensure that even some very powerful adversaries attacking over many years have only a small probability of causing irrecoverable damage before being detected.

Emerging Web services, such as email, photo sharing, and web site archives, must preserve large volumes of quickly accessible data indefinitely into the future. The costs of doing so often determine whether the service is economically viable. We make the case that these applications' demands on large scale storage systems over long time horizons require us to reevaluate traditional system designs. We examine threats to long-lived data from an end-to-end perspective, taking into account not just hardware and software faults but also faults due to humans and organizations. We present a simple model of long-term storage failures that helps us reason about various strategies for addressing some of these threats. Using this model we show that the most important strategies for increasing the reliability of long-term storage are detecting latent faults quickly, automating fault repair to make it cheaper and faster, and increasing the independence of data replicas.

A growing number of online services, such as Google, Yahoo!, and Amazon, are starting to

digital preservation, archival storage, repository We are in the midst of an unprecedented transformation from physical to virtual assets. Online contracts, digital photographs, digitized movies, music, technical journals, corporate records, web sites, and government documents are just a few examples of valuable digital assets that organizations would often like to preserve for long periods of time-- not just for years, but for decades or even forever. Unfortunately, long-term preservation remains a huge challenge due to the unusual nature of the threats from which it suffers compared to traditional (shorter-term) storage applications. Our goal in this paper is to describe how these environments differ and to acquaint the dependability community with some of the challenges in building archival storage systems. We give some guidelines for an alternative storage architecture, much of which is being implemented at the British Library, and we conclude with some suggestions for initial research topics to be tackled in this area.

Peer-to-peer systems in which the peers are truly autonomous have valuable properties, including resistance to certain forms of organizational failure and legal attack. Unfortunately, they can be vulnerable to malign peers. In the context of the LOCKSS system, a peer-to-peer digital preservation system for e-journals, we describe a set of techniques that enable a large population of autonomous peers to resist attack by a substantial minority of malign peers endowed with unlimited computational resources. LOCKSS peers are able to detect attacks and alert the community of peer operators before damage becomes irreversible. These techniques include rate limitation and making peers "pay" certain costs by demanding proofs of effort from them.