Virtual disks are the main form of storage in today’s virtual machine environments. They offer many attractive features, including whole system versioning, isolation, and mobility, that are absent from current file systems. Unfortunately, the low-level interface of virtual disks is very coarse-grained, forcing all-or-nothing whole system rollback, and opaque, offering no practical means of sharing. These problems impose serious limitations on virtual disks ’ usability, security, and ease of management. To overcome these limitations, we offer Ventana, a virtualization aware file system. Ventana combines the filebased storage and sharing benefits of a conventional distributed file system with the versioning, mobility, and access control features that make virtual disks so compelling. 1

Abstract: This paper describes Haystack, an object storage system optimized for Facebook’s Photos application. Facebook currently stores over 260 billion images, which translates to over 20 petabytes of data. Users upload one billion new photos (∼60 terabytes) each week and Facebook serves over one million images per second at peak. Haystack provides a less expensive and higher performing solution than our previous approach, which leveraged network attached storage appliances over NFS. Our key observation is that this traditional design incurs an excessive number of disk operations because of metadata lookups. We carefully reduce this per photo metadata so that Haystack storage machines can perform all metadata lookups in main memory. This choice conserves disk operations for reading actual data and thus increases overall throughput. 1

As storage systems evolve, the block-based design of today’s disks is becoming inadequate. As an alternative, objectbased storage devices (OSDs) offer a view where the disk manages data layout and keeps track of various attributes about data objects. By moving functionality that is traditionally the responsibility of the host OS to the disk, it is possible to improve overall performance and simplify management of a storage system. The capabilities of OSDs will also permit performance improvements in parallel file systems, such as further decoupling metadata operations and thus reducing metadata server bottlenecks. In this work we present an implementation of the Parallel Virtual File System (PVFS) integrated with a software emulator of an OSD and describe an infrastructure for client access. Even with the overhead of emulation, performance is comparable to a traditional server-fronted implementation, demonstrating that serverless parallel file systems using OSDs are an achievable goal. 1.

Object based storage devices (OSDs) elevate the level of abstraction presented to clients, thereby permitting them to offer methods for managing, sharing, and securing information that go beyond those offered by block-based stores. The Object-Oriented Storage System (O2S2) architecture presented and evaluated in this paper implements a virtualization service to provide object-based storage in a virtualized environment. This service provides a virtual object-based storage device (vOSD) to virtual machines. The use of vOSDs permits the service provider, i.e., the vOSD storage domain, to offer to guest virtual machines new methods for resource management and consolidation, without requiring the purchase of physical storage devices that faithfully implement OSD functionality. Methods demonstrated in this paper include improved support for access control and for heterogeneity of storage devices. Advantages derived from such methods also include reduced complexity for end clients, i.e., guest VMs. A prototype PVFS-based O2S2 implementation demonstrates that its enhanced services can be provided at low cost, enabled in part by the efficient utilization of otherwise idle domain resources. 1

Developers often describe testing as being tedious and boring. Our work challenges this notion. We describe tools and methodologies crafted to test object-based storage devices (OSDs) for correctness and compliance with the T10 OSD standard. Special consideration is given to testing the security model of an OSD implementation. Some work was also carried out on building OSD benchmarks. This work can serve as a basis for a general-purpose benchmark suite for OSDs in the future, as more OSD implementations emerge. The tool described here has been used to verify object-disks built by Seagate and IBM Research. 1

Abstract not found

As the world becomes digital, we are in ever greater danger of losing business, scientific, artistic, cultural, and personal assets. The threat of such a digital dark age stems from the fact that— unlike physical records that may survive decades, centuries, or even longer without advanced planning—digital records will not survive without planning and diligence. Everything needed to keep digital records viable will become obsolete, including hardware, software, processes, and formats. Consequently, digital preservation environments are needed to ensure the ability to access valuable digital records decades from now and, more significantly, to ensure the interpretability of the records once accessed. We describe Preservation DataStores, an innovative storage architecture that facilitates robust and optimized preservation environments. It is a layered architecture that builds upon open standards, including Open Archival Information System, XAM (Extensible Access Method), and Object-based Storage Device. We also describe the integration of Preservation DataStores with existing file systems and archives and discuss some design and implementation issues. We are developing Preservation DataStores as an infrastructure component of the European Union CASPAR (Cultural, Artistic and Scientific knowledge for Preservation, Access and Retrieval) project, where it will be used to preserve scientific, cultural, and artistic data.

Abstract—Today’s distributed file system architectures scale well to large amounts of data. Their performance, however, is often limited by their metadata server. In this paper, we reconsider the database backend of the metadata server and propose a design that simplifies implementation and enhances performance. In particular, we argue that the concept of log-structured merge (LSM) trees is a better foundation for the storage layer of a metadata server than the traditionally used B-trees. We present BabuDB, a database that relies on LSM-tree-like index structures, and describe how it stores file system metadata. We show that our solution offers better scalability and performance than equivalent ext4 and BerkeleyDB-based metadata server implementations. Our experiments include real-world metadata traces from a Linux kernel build and an IMAP mail server. Results show that BabuDB is up to twice as fast as the ext4-based backend and outperforms a BerkeleyDB setup by an order of magnitude. I.

Abstract — The rapid growth of data and data sharing have been driven an evolution in distributed storage infrastructure. The need for sensitive data protection and the capacity to handle massive data sets have encouraged the research and development of secure and scalable storage systems. This paper identifies major security issues and requirements of data protection related to distributed data storage systems. We classify the security services and techniques in existing or proposed storage systems. We then discuss potential research topics and future trends.

c t i v it y e p o r t 2009 Table of contents