We propose a new design for highly concurrent Internet services, whichwe call the staged event-driven architecture (SEDA). SEDA is intended

An Architecture for Highly Concurrent, Well-Conditioned Internet Services by Matthew David Welsh Doctor of Philosophy in Computer Science University of California at Berkeley Professor David Culler, Chair This dissertation presents an architecture for handling the massive concurrency and load conditioning demands of busy Internet services. Our thesis is that existing programming models and operating system structures do not adequately meet the needs of complex, dynamic Internet servers, which must support extreme concurrency (on the order of tens of thousands of client connections) and experience load spikes that are orders of magnitude greater than the average. We propose a new software framework, called the staged event-driven architecture (or SEDA), in which applications are constructed as a network of event-driven stages connected with explicit queues. In this model, each stage embodies a robust, reusable software component that performs a subset of request processing. By performing admission control on each event queue, the service can be well-conditioned to load, preventing resources from being overcommitted when demand exceeds service capacity. SEDA employs dynamic control to tune runtime parameters (such as the scheduling parameters of each stage) automatically, as well as to manage load, for example, by performing adaptive load shedding.

This position paper makes the case that overload management should be a critical design goal for Internet-based systems and services. Few Internet service designs take overload into account, treating the problem as one of capacity planning rather than engineering the service to behave gracefully under extreme load. We argue that the right approach to overload management is to explicitly signal overload conditions to the application, allowing it to participate in resource management decisions. Furthermore, we claim that feedback-driven control, rather than static resource limits, should be the basis for detecting and controlling overload. We present a feedback-driven approach to overload control based on the staged event-driven architecture (SEDA) model for Internet service design. This approach makes use of adaptive admission controllers for meeting administrator-specified performance targets, such as 90th percentile response time. We demonstrate the use of these overload control mechanisms in two applications: a complex Web-based e-mail service, and a dynamic Web server benchmark.

This work is informed by two important schools of thought in distributed computing: computational grids and web services. We use the main ideas from these disciplines to give a conceptual framework for conglomerate mathematical systems which are distributed over the internet.

Distributed systems with high availability requirements have to support some form of dynamic reconfiguration. This means that they must provide the ability to be maintained or upgraded without being taken off-line. This thesis addresses the dynamic reconfiguration of distributed applications that run on top of an object-middleware infrastructure. In this context, a system configuration is defined as a structure of software entities at application-level. Dynamic reconfiguration entails operations for the replacement, migration, creation and removal of these entities at run-time. This thesis proposes an approach to dynamic reconfiguration for applications built using object-middleware and realizes this approach with an architecture and design of middleware support for reconfiguring CORBA-based distributed systems at run-time. ii Table of contents 1

i Distributed systems with high availability requirements have to support some form of dynamic reconfiguration. This means that they must provide the ability to be maintained or upgraded without being taken off-line. This thesis addresses the dynamic reconfiguration of distributed applications that run on top of an object-middleware infrastructure. In this context, a system configuration is defined as a structure of software entities at application-level. Dynamic reconfiguration entails operations for the replacement, migration, creation and removal of these entities at run-time. This thesis proposes an approach to dynamic reconfiguration for applications built using object-middleware and realizes this approach with an architecture and design of middleware support for reconfiguring CORBA-based distributed systems at run-time.