Abstract. This paper addresses the problem of dynamic system reconfiguration and resource sharing for a set of applications in large-scale services environments. It presents a decentralized application placement scheme that dynamically provisions enterprise applications with heterogeneous resource requirements. Potential benefits, including improved scalability, resilience, and continuous adaptation to external events, motivate a decentralized approach. In our design, all nodes run a placement controller independently and asynchronously, which periodically reallocates a node’s local resources to applications based on state information from a fixed number of neighbors. Compared with a centralized solution, our placement scheme incurs no additional synchronization costs. We show through simulations that decentralized placement can achieve accuracy close to that of state-of-the-art centralized placement schemes (within 4% in a specific scenario). In addition, we report results on scalability and transient behavior of the system. 1

Most research on Distributed Hash Tables (DHTs) assumes ephemeral, lightly loaded deployments. Each node has a lifetime of a few hours and initiates a lookup once every few seconds or minutes. However, in giant internet data centers, each node has a lifetime of weeks or months and initiates hundreds or thousands of lookups every second. In such an environment, one-hop DHTs are superior to multi-hop DHTs. They use lookup bandwidth more efficiently. We qualify conflicting research to show that a single onehop DHT can indeed scale to at least a few hundred thousand nodes in stable, high-capacity enterprise networks. Two new designs are presented: One Hop Sites (1HS), a high-capacity DHT tailored for site redundancy; and the One Hop Federation (1HF), a global, hierarchic DHT that resolves an open latency problem. For both, the analysis a) confirms linear scalability to at least a few hundred thousand nodes and b) identifies the most sensitive design parameters.