We propose a new design for highly concurrent Internet services, whichwe call the staged event-driven architecture (SEDA). SEDA is intended

Web server performance has steadily improved since the inception of the World Wide Web. We observe performance gains of two orders of magnitude between the original process-based Web servers and today's threaded Web servers. Commercial and academic Web servers achieved much of these gains using new or improved event-notification mechanisms and techniques to eliminate reading and copying data, both of which required new operating system primitives. More recently, experimental and production Web servers began integrating HTTP processing in the TCP/IP stack and providing zero copy access to a kernel-managed cache. These kernelmode Web servers improved upon newer user-mode Web servers by a factor of two to six.

Building highly concurrent systems, such as large-scale Internet services, requires managing many information flows at once and maintaining peak throughput when demand exceeds resource availability. In addition, any platform supporting Internet services must provide high availability and be able to cope with burstiness of load. Many approaches to building concurrent systems have been proposed, which generally fall into the two categories of threaded and event-driven programming. We propose that threads and events are actually on the ends of a design spectrum, and that the best implementation strategy for these applications is somewhere in between. We present a general-purpose design framework for building highly concurrent systems, based on three design components --- tasks, queues, and thread pools --- which encapsulate the concurrency, performance, fault isolation, and software engineering benefits of both threads and events. We present a set of design patterns that can be applied to...

We argue that existing OS designs are ill-suited for the needs of Internet service applications. These applications demand massive concurrency (supporting a large number of requests per second) and must be well-conditioned to load (avoiding degradation of performance and predictability when demand exceeds capacity). The transparency and virtualization provided by existing operating systems leads to limited concurrency and lack of control over resource usage. We claim that Internet services would be far better supported by operating systems by reconsidering the role of resource virtualization. We propose a new design for server applications, the staged event-driven architecture (SEDA). In SEDA, applications are constructed as a set of eventdriven stages separated by queues. We present the SEDA architecture and its consequences for operating system design. 1.

An Architecture for Highly Concurrent, Well-Conditioned Internet Services by Matthew David Welsh Doctor of Philosophy in Computer Science University of California at Berkeley Professor David Culler, Chair This dissertation presents an architecture for handling the massive concurrency and load conditioning demands of busy Internet services. Our thesis is that existing programming models and operating system structures do not adequately meet the needs of complex, dynamic Internet servers, which must support extreme concurrency (on the order of tens of thousands of client connections) and experience load spikes that are orders of magnitude greater than the average. We propose a new software framework, called the staged event-driven architecture (or SEDA), in which applications are constructed as a network of event-driven stages connected with explicit queues. In this model, each stage embodies a robust, reusable software component that performs a subset of request processing. By performing admission control on each event queue, the service can be well-conditioned to load, preventing resources from being overcommitted when demand exceeds service capacity. SEDA employs dynamic control to tune runtime parameters (such as the scheduling parameters of each stage) automatically, as well as to manage load, for example, by performing adaptive load shedding.

We propose a new design for highly-concurrent server applications such as Internet services. This design, the staged event-driven architecture (SEDA), is intended to support massive concurrency demands for a wide range of applications. In SEDA, applications are constructed as a set of event-driven stages separated by queues. This design allows services to be well-conditioned to load, preventing resources from being overcommitted when demand exceeds service capacity. Decomposing services into a set of stages enables modularity and code reuse, as well as the development of debugging tools for complex event-driven applications. We present the SEDA design, as well as Sandstorm, an Internet services platform based on this architecture. We evaluate the use of Sandstorm through two applications: A simple HTTP server benchmark and a packet router for the Gnutella peer-to-peer file sharing network. 1 Introduction The Internet presents a systems problem of unprecedented scale: that of supporti...

copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. The author reserves all other publication and other rights in association with the copyright in the thesis, and except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatever without the author’s permission. Date:

Abstract not found