Two recent techniques for multicast or broadcast delivery of streaming media can provide immediate service to each client request, yet achieve considerable client stream sharing which leads to significant server and network bandwidth savings. This paper considers (1) how well these recently proposed techniques perform relative to each other, and (2) whether there are new practical delivery techniques that can achieve better bandwidth savings than the previous techniques over a wide range of client request rates. The principal results are as follows. First, the recent partitioned dynamic skyscraper technique is adapted to provide immediate service to each client request more simply and directly than the original dynamic skyscraper method. Second, at moderate to high client request rates, the dynamic skyscraper method has required server bandwidth that is significantly lower than the recent optimized stream tapping/patching/controlled multicast technique. Third, the minimum required ser...

Emerging large-scale distributed storage systems are faced with the task of distributing petabytes of data among tens or hundreds of thousands of storage devices. Such systems must evenly distribute data and workload to efficiently utilize available resources and maximize system performance, while facilitating system growth and managing hardware failures. We have developed CRUSH, a scalable pseudorandom data distribution function designed for distributed object-based storage systems that efficiently maps data objects to storage devices without relying on a central directory. Because large systems are inherently dynamic, CRUSH is designed to facilitate the addition and removal of storage while minimizing unnecessary data movement. The algorithm accommodates a wide variety of data replication and reliability mechanisms and distributes data in terms of userdefined policies that enforce separation of replicas across failure domains. 1

Abstract Scalable storage architectures allow for the addition ofdisks to increase storage capacity and/or bandwidth. In its general form, disk scaling also refers to disk removals wheneither capacity needs to be conserved or old disk drives are retired. Assuming random placement of blocks on multiplenodes of a continuous media server, our optimization objective is to redistribute a minimum number of media blocksafter disk scaling. This objective should be met under two restrictions. First, uniform distribution and hence a bal-anced load should be ensured after redistribution. Second, the redistributed blocks should be retrieved at the normalmode of operation in one disk access and through low complexity computation. We propose a technique that meets theobjective, while we prove that it also satisfies both restrictions. The SCADDAR approach is based on using a seriesof R EMAP functions which can derive the location of a newblock using only its original location as a basis. 1.

The performance of streaming media servers has been limited due to the dual requirements of high throughput and low memory use. Although disk throughput has been enjoying a 40% annual increase, slower improvements in disk access times necessitate the use of large DRAM buffers to improve the overall streaming throughput. MEMS-based storage is an exciting new technology that promises to bridge the widening performance gap between DRAM and disk-drives in the memory hierarchy. This paper explores the impact of integrating these devices into the memory hierarchy on the class of streaming media applications. We evaluate the use of MEMS-based storage for buffering and caching streaming data. We also show how a bank of k MEMS devices can be managed in either configuration and that they can provide a k-fold improvement in both throughput and access latency. An extensive analytical study shows that using MEMS storage can reduce the buffering cost and improve the throughput of streaming servers significantly.

Abstract—Allocation of data to parallel disk using redundant storage and random placement of blocks can be exploited to achieve low access delays. New algorithms are proposed which improve the previously known shortest queue algorithm by systematically exploiting that scheduling decisions can be deferred until a block access is actually started on a disk. These algorithms are also generalized for coding schemes with low redundancy. Using extensive simulations, practically important quantities are measured which have so far eluded an analytical treatment: The delay distribution when a stream of requests approaches the limit of the sytem capacity, the system efficiency for parallel disk applications with bounded prefetching buffers, and the combination of both for mixed traffic. A further step toward practice is taken by outlining the system design for: automatically load-balanced parallel hard-disk array. Additional algorithmic measures are proposed for that allow variable sized blocks, seek time reduction, fault tolerance, inhomogeneous systems, and flexible priorization schemes. Index Terms—Parallel disks, lazy scheduling, asynchronous, random redundant storage, duplicate allocation, soft real time, bipartite matching, queuing theory. 1

distinguishes itself from other similar research efforts in the following: . complete distribution with all nodes running identical software and no single points of failure; . efficient online scalability allowing disks to be added or removed without interrupting CM streams; . synchronization of several streams of audio, video, or both within one frame (1/30 second); . independence from media types; . compliance with industry standards; . selective retransmission protocol; and . multithreshold buffering flow-control mechanism to support variable bit-rate (VBR) media. Yima is also a complete end-to-end system that uses an IP network with several supportable client types. This feature distinguishes it from previous research that focused heavily on server design. SYSTEM ARCHITECTURE Figure 1 shows the overall Yima system architecture. In our prototype implementation, the server consists of an eight-way cluster of rack-mountable Dell PowerEdge 1550 Pentium III 866-MH

Abstract: Presently, digital continuous media (CM) are well established as an integral part of many applications. In recent years, a considerable amount of research has focused on the efficient retrieval of such media. Scant attention has been paid to servers that can record such streams in real time. However, more and more devices produce direct digital output streams. Hence, the need arises to capture and store these streams with an efficient data stream recorder that can handle both recording and playback of many streams simultaneously and provide a central repository for all data. We propose a design for a large scale data stream recorder. Our goal was to introduce a unified architecture that integrates multi-stream recording and retrieval in a coherent manner. The discussion raises practical issues such as support for multizone disk drives, variable bit rate media, and disk drives that have a different write than read bandwidth. We provide initial solutions for some issues while others will need to be investigated further. 1

Large-scale continuous media (CM) system implementations require scalable servers most likely built from clusters of storage nodes. Across such nodes random data placement is an attractive alternative to the traditional round-robin striping. One benefit of random placement is that additional nodes can be added with low data-redistribution overhead such that the system remains load balanced. One of the challenges in this environment is the implementation of a retransmission-based error control (RBEC) technique. Because data is randomly placed, a client may not know which server node to ask for a lost packet retransmission.

The wide availability of broadband networking technologies such as cable modems and DSL coupled with the growing popularity of the Internet has led to a dramatic increase in the availability and the use of online streaming media. With the “last mile ” network bandwidth no longer a constraint, the bottleneck for video streaming has been pushed closer to the server. Streaming high quality audio and video to a myriad of clients imposes significant resource demands on the server. In this work, we propose a demand adaptive and locality aware (DALA) clustered media server architecture that can dynamically allocate resources to adapt to changing demand and also maximize the number of clients serviced by the server cluster. Moreover, our design exploits temporal locality among requests by dispatching newly arriving requests to servers that are already servicing prior requests for those objects, thereby extracting the benefits of locality. We explore the efficacy of the DALA clustered architecture using simulations. Our simulation results show that DALA is highly adaptive, exhibits significant performance gains when compared to static schemes, and has a low system overhead. Our results demonstrate that DALA is a simple, yet effective approach for designing clustered media servers. 1

Data-intensive Grid applications need access to large datasets that may each be replicated on different resources. Minimizing the overhead of transferring these datasets to the resources where the applications are executed requires that appropriate computational and data resources be selected. In this paper, we introduce a heuristic for the selection of resources based on a solution to the Set Covering Problem (SCP). We then pair this mapping heuristic with the well-known MinMin scheduling algorithm and conduct performance evaluation through extensive simulations.