Results 1 - 10
of
99
CoolStreaming/DONet: A Data-driven Overlay Network for Peer-to-Peer Live Media Streaming
- in IEEE Infocom
, 2005
"... This paper presents DONet, a Data-driven Overlay Network for live media streaming. The core operations in DONet are very simple: every node periodically exchanges data availability information with a set of partners, and retrieves unavailable data from one or more partners, or supplies available dat ..."
Abstract
-
Cited by 245 (31 self)
- Add to MetaCart
This paper presents DONet, a Data-driven Overlay Network for live media streaming. The core operations in DONet are very simple: every node periodically exchanges data availability information with a set of partners, and retrieves unavailable data from one or more partners, or supplies available data to partners. We emphasize three salient features of this data-driven design: 1) easy to implement, as it does not have to construct and maintain a complex global structure; 2) efficient, as data forwarding is dynamically determined according to data availability while not restricted by specific directions; and 3) robust and resilient, as the partnerships enable adaptive and quick switching among multi-suppliers. We show through analysis that DONet is scalable with bounded delay. We also address a set of practical challenges for realizing DONet, and propose an efficient member- and partnership management algorithm, together with an intelligent scheduling algorithm that achieves real-time and continuous distribution of streaming contents.
Lightweight probabilistic broadcast
- ACM Transaction on Computer Systems
, 2003
"... The growing interest in peer-to-peer applications has underlined the importance of scalability in modern distributed systems. Not surprisingly, much research effort has been invested in gossip-based broadcast protocols. These trade the traditional strong reliability guarantees against very good “sca ..."
Abstract
-
Cited by 199 (28 self)
- Add to MetaCart
The growing interest in peer-to-peer applications has underlined the importance of scalability in modern distributed systems. Not surprisingly, much research effort has been invested in gossip-based broadcast protocols. These trade the traditional strong reliability guarantees against very good “scalability” properties. Scalability is in that context usually expressed in terms of throughput and delivery latency, but there is only little work on how to reduce the overhead of membership management at large scale. This paper presents Lightweight Probabilistic Broadcast (lpbcast), a novel gossip-based broadcast algorithm which preserves the inherent throughput scalability of traditional gossip-based algorithms and adds a notion of membership management scalability: every process only knows a random subset of fixed size of the processes in the system. We formally analyze our broadcast algorithm in terms of scalability with respect to the size of individual views, and compare the analytical results both with simulations and concrete measurements.
The Peer Sampling Service: Experimental Evaluation of Unstructured Gossip-Based Implementations
- In Middleware ’04: Proceedings of the 5th ACM/IFIP/USENIX international conference on Middleware
, 2004
"... Abstract. In recent years, the gossip-based communication model in large-scale distributed systems has become a general paradigm with important applications which include information dissemination, aggregation, overlay topology management and synchronization. At the heart of all of these protocols l ..."
Abstract
-
Cited by 124 (28 self)
- Add to MetaCart
Abstract. In recent years, the gossip-based communication model in large-scale distributed systems has become a general paradigm with important applications which include information dissemination, aggregation, overlay topology management and synchronization. At the heart of all of these protocols lies a fundamental distributed abstraction: the peer sampling service. In short, the aim of this service is to provide every node with peers to exchange information with. Analytical studies reveal a high reliability and efficiency of gossip-based protocols, under the (often implicit) assumption that the peers to send gossip messages to are selected uniformly at random from the set of all nodes. In practice—instead of requiring all nodes to know all the peer nodes so that a random sample could be drawn—a scalable and efficient way to implement the peer sampling service is by constructing and maintaining dynamic unstructured overlays through gossiping membership information itself. This paper presents a generic framework to implement reliable and efficient peer sampling services. The framework generalizes existing approaches and makes it easy to introduce new ones. We use this framework to explore and compare several implementations of our abstract scheme. Through extensive experimental analysis, we show that all of them lead to different peer sampling services none of which is uniformly random. This clearly renders traditional theoretical approaches invalid, when the underlying peer sampling service is based on a gossip-based scheme. Our observations also help explain important differences between design choices of peer sampling algorithms, and how these impact the reliability of the corresponding service. 1
CYCLON: Inexpensive Membership Management for Unstructured P2P Overlays
- Journal of Network and Systems Management
, 2005
"... Unstructured overlays form an important class of peer-to-peer networks, notably when content-based searching is at stake. The construction of these overlays, which is essentially a membership management issue, is crucial. Ideally, the resulting overlays should have low diameter and be resilient to m ..."
Abstract
-
Cited by 120 (15 self)
- Add to MetaCart
Unstructured overlays form an important class of peer-to-peer networks, notably when content-based searching is at stake. The construction of these overlays, which is essentially a membership management issue, is crucial. Ideally, the resulting overlays should have low diameter and be resilient to massive node failures, which are both characteristic properties of random graphs. In addition, they should be able to deal with a high node churn (i.e., expect high-frequency membership changes). Inexpensive membership management while retaining random-graph properties is therefore important. In this paper, we describe a novel gossip-based membership management protocol that meets these requirements. Our protocol is shown to construct graphs that have low diameter, low clustering, highly symmetric node degrees, and that are highly resilient to massive node failures. Moreover, we show that the protocol is highly reactive to restoring randomness when a large number of nodes fail. KEY WORDS: Membership management; peer-to-peer; epidemic/gossiping protocols; unstructured overlays; random graphs.
Probabilistic Reliable Dissemination in Large-Scale Systems
- IEEE Transactions on Parallel and Distributed Systems
, 2001
"... The growth of the Internet raises new challenges for the design of distributed systems and applications. In the context of group communication protocols, gossip-based schemes have attracted interest as they are scalable, easy to deploy, and resilient to network and process failures. However, tradi ..."
Abstract
-
Cited by 112 (18 self)
- Add to MetaCart
The growth of the Internet raises new challenges for the design of distributed systems and applications. In the context of group communication protocols, gossip-based schemes have attracted interest as they are scalable, easy to deploy, and resilient to network and process failures. However, traditional gossip-based protocols have two major drawbacks: 1) They rely on each peer having knowledge of the global membership and 2) being oblivious to the network topology, they can impose a high load on network links when applied to wide-area settings. In this paper, we provide a theoretical analysis of gossip-based protocols which relates their reliability to key system parameters (system size, failure rates, and number of gossip targets). The results provide guidelines for the design of practical protocols. In particular, they show how reliability can be maintained while alleviating drawback 1) by providing each peer with only a small subset of the total membership information and drawback 2) by organizing members into a hierarchical structure that reflects their proximity according to some network-related metric. We validate the analytical results by simulations and verify that the hierarchical gossip protocol considerably reduces the load on the network compared to the original, nonhierarchical protocol.
From Epidemics to Distributed Computing
- IEEE Computer
, 2004
"... Abstract — Epidemic algorithms have been recently recognized as robust and scalable means to disseminate information in large-scale settings. Information is disseminated reliably in a distributed system the same way an epidemic would be propagated throughout a group of individuals: each process of t ..."
Abstract
-
Cited by 60 (4 self)
- Add to MetaCart
Abstract — Epidemic algorithms have been recently recognized as robust and scalable means to disseminate information in large-scale settings. Information is disseminated reliably in a distributed system the same way an epidemic would be propagated throughout a group of individuals: each process of the system chooses random peers to whom it relays the information it has received. The underlying peer-to-peer communication paradigm is the key to the scalability of the dissemination scheme. Epidemic algorithms have been studied theoretically and their analysis is built on sound mathematical foundations. Although promising, their general applicability to large scale distributed systems has yet to go through addressing many issues. These constitute an exciting research agenda. Index Terms — Scalability, peer-to-peer, epidemics, information
Newscast Computing
, 2003
"... Monitoring large computer networks often involves aggregation of various sorts of data that are distributed across network components. Finding extreme values, counting discrete observations or computing an average or a sum of some parameter values are typical examples of such "background" activities ..."
Abstract
-
Cited by 52 (9 self)
- Add to MetaCart
Monitoring large computer networks often involves aggregation of various sorts of data that are distributed across network components. Finding extreme values, counting discrete observations or computing an average or a sum of some parameter values are typical examples of such "background" activities that provide input to monitoring systems. Another aspect of network management is fast and reliable information dissemination, like propagation of alarm signals.
Epidemic-Style Proactive Aggregation in Large Overlay Networks
- In Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS’04
, 2004
"... Aggregation---that is, the computation of global properties like average or maximal load, or the number of nodes--- is an important basic functionality in fully distributed environments. In many cases---which include protocols responsible for self-organization in large-scale systems and collaborativ ..."
Abstract
-
Cited by 43 (14 self)
- Add to MetaCart
Aggregation---that is, the computation of global properties like average or maximal load, or the number of nodes--- is an important basic functionality in fully distributed environments. In many cases---which include protocols responsible for self-organization in large-scale systems and collaborative environments---it is useful if all nodes know the value of some aggregates continuously. In this paper we present and analyze novel protocols capable of providing this service. The proposed anti-entropy aggregation protocols compute different aggregates of component properties like extremal values, average and counting. Our protocols are inspired by the anti-entropy epidemic protocol where random pairs of databases periodically resolve their differences. In the case of aggregation, resolving difference is generalized to an arbitrary (numeric) computation based on the states of the two communicating peers. The advantage of this approach is that it is proactive and "democratic", which means it has no performance bottlenecks, and the approximation of the aggregates is present continuously at all nodes. These properties make our protocol suitable for implementing e.g. collective decision making or automatic system maintenance based on global information in a fully distributed fashion. As our main contribution we provide fundamental theoretical results on the proposed averaging protocol.
Large-Scale Newscast Computing on the Internet
, 2002
"... This paper introduces the newscast model of computation for large-scale computing on the Internet. The engine realizing this model is a lazy fully distributed information propagation protocol among the participants which is responsible for membership management and communication. It maintains a cons ..."
Abstract
-
Cited by 39 (14 self)
- Add to MetaCart
This paper introduces the newscast model of computation for large-scale computing on the Internet. The engine realizing this model is a lazy fully distributed information propagation protocol among the participants which is responsible for membership management and communication. It maintains a constantly changing communication graph over the participants. This graph has useful emergent properties like small diameter and sufficiently random structure without deploying special purpose protocols to achieve these properties. For adding a new participant only the address of an arbitrary member is needed and for removal no action is necessary. We provide theoretical and empirical evidence that besides being simple and lightweight our newscast computing engine is extremely scalable and robust. We also suggest some interesting application areas including information dissemination, monitoring of large systems, resource sharing and efficient multicasting.
Exploiting semantic proximity in peer-to-peer content searching
- In 10th International Workshop on Future Trends in Distributed Computing Systems (FTDCS 2004), Suzhu
, 2004
"... A lot of recent work has dealt with improving performance of content searching in peer-to-peer file sharing systems. In this paper we attack this problem by modifying the overlay topology describing the peer relations in the system. More precisely, we create a semantic overlay, linking nodes that ar ..."
Abstract
-
Cited by 36 (10 self)
- Add to MetaCart
A lot of recent work has dealt with improving performance of content searching in peer-to-peer file sharing systems. In this paper we attack this problem by modifying the overlay topology describing the peer relations in the system. More precisely, we create a semantic overlay, linking nodes that are “semantically close”, by which we mean that they are interested in similar documents. This semantic overlay provides the primary search mechanism, while the initial peer-to-peer system provides the fail-over search mechanism. We focus on implicit approaches for discovering semantic proximity. We evaluate and compare three candidate methods, and review open questions. 1.

