Developing extensible, robust, and efficient distributed systems is a complex task. To help alleviate this complexity, we have developed the ADAPTIVE Service eXecutive (ASX) framework. ASX is an object-oriented framework composed of automated tools and reusable components. These tools and components help to simplify the development, configuration, and reconfiguration of applications in a distributed environment. Using the ASX framework, the services in the applications may be updated and extended without modifying, recompiling, relinking, or restarting the applications at run-time. This paper describes the features and objectoriented architecture of the ASX framework. It also describes how the ASX framework has been used to develop a highly modular, reusable, and dynamically reconfigurable family of distributed system management applications. 1 Introduction Mapping distributed application services flexibly and efficiently onto host processes and threads is a challenging task. Selectin...

Parallel processing has been proposed as a means of improving network protocol throughput. Several different strategies have been taken towards parallelizing protocols. A relatively popular approach is packet-level parallelism, where packets are distributed across processors. This paper provides an experimental performance study of packet-level parallelism on a contemporary sharedmemory multiprocessor. We examine several unexplored areas in packet-level parallelism and investigate how various protocol structuring and implementation techniques can affect performance. We study TCP/IP and UDP/IP protocol stacks, implemented with a parallel version of the x-kernel running in user space on Silicon Graphics multiprocessors. Our results show that only limited packet-level parallelism can be achieved within a single connection under TCP, but that using multiple connections can improve available parallelism. We also demonstrate that packet ordering plays a key role in determining single-connection TCP performance, that careful use of locks is a necessity, and that selective exploitation of caching can improve throughput. We also describe experiments that compare parallel protocol performance on two generations of a parallel machine and show how computer architectural trends can influence performance. 1

Techniques for avoiding the high memory overheads found on many modern shared-memory multiprocessors are of increasing importance in the development of high-performance multiprocessor protocol implementations. One such technique is processor-cache affinity scheduling, which can significantly lower packet latency and substantially increase protocol processing throughput [30]. In this paper, we evaluate several aspects of the effectiveness of affinity-based scheduling in multiprocessor network protocol processing, under packet-level and connection-level parallelization approaches. Specifically, we evaluate the performance of the scheduling technique 1) when a large number of streams are concurrently supported, 2) when processing includes copying of uncached packet data, 3) as applied to send-side protocol processing, and 4) in the presence of stream burstiness and source locality, two well-known properties of network traffic. We find that affinity-based scheduling performs well under the...

: This paper analyzes the efficiency of various high performance implementation techniques for the communication system of UNIX workstations. Using an Open System implies that a certain compatibility level is required from the protocol, user interface, and implementation framework. These constraints limit the opportunities to design a high performance communication system. We have designed an experimental platform around the TCP/IP protocol suite, using the STREAMS environment. A BSD TCP/IP stack and a classic STREAMS based TCP/IP stack serve as reference implementations for performance comparisons. We explain why the efficiency of some high performance implementation techniques we applied to this platform is limited. The impacts of the hardware architecture, of the operating system, and of the communication stack architecture on performances are analyzed. It is shown that the efficiency of data transmission would benefit from more simplicity and more synchronism in the communication e...

Security and privacy are growing concerns in the Internet community, due to the Internet's rapid growth and the desire to conduct business over it safely. This desire has led to the advent of several proposals for security standards, such as secure IP, secure HTTP, and the Secure Socket Layer. All of these standards propose using cryptographic protocols such as DES and RSA. Thus, the need to use encryption protocols is increasing. Shared-memory multiprocessors make attractive server platforms, for example as secure World-Wide Web servers. These machines are becoming more common, as shown by recent vendor introductions of platforms such as SGI's Challenge, Sun's SPARCCenter, and DEC's AlphaServer. The spread of these machines is due both to their relative ease of programming and their good price/performance. This paper is an experimental performance study that examines how encryption protocol performance can be improved by using parallelism. We show linear speedup for several different ...

We explore processor-cache affinity scheduling of parallel network protocol processing, in a setting in which protocol processing executes on a shared-memory multiprocessor concurrently with a general workload of non-protocol activity. We find affinity-based scheduling can significantly reduce the communication delay associated with protocol processing, enabling the host to support a greater number of concurrent streams and to provide higher maximum throughput to individual streams. In addition, we compare the performance of two parallelization alternatives, Locking and Independent Protocol Stacks (IPS), with very different caching behaviors. We find that IPS (which maximizes cache affinity) delivers much lower message latency and significantly higher message throughput capacity, yet exhibits less robust response to intra-stream burstiness and limited intra-stream scalability. 1 Introduction In many modern computer architectures, there is a significant difference in the amount of ti...

In this paper, we explore the benefits of processor cache affinity scheduling of parallelized network protocol processing. We find that affinity scheduling, which has not previously been shown to be of significant benefit to common applications, can provide large performance gain in the context of parallelized protocol processing. We conduct a set of multiprocessorexperiments designed to measure packet processing time in a UDP/IP/FDDI protocol stack in the x-kernel on an SGI Challenge XL multiprocessor. These measurements are then used to parameterize a combined simulation/analytic model of multiprocessor protocol processing. Our simulation results show that affinity scheduling can significantly reduce message delay associated with protocol processing, allowing a host to support a greater number of concurrent streams, to provide a higher maximum throughput to individual streams, and to decrease the end-to-end latency seen by an application. We find the reduction in end-to-end l...

In this paper, we present further results in processor-cache affinity scheduling of parallel network protocol processing, in a setting in which protocol processing executes on the multiprocessor host concurrently with a general workload of non-protocol activity. In earlier work [31, 32] we evaluated affinity-based scheduling of receive-side protocol processing under two parallelization approaches: Locking and Independent Protocol Stacks (IPS). In this work, we i) evaluate affinity-based scheduling of send-side UDP/IP/FDDI processing, ii) examine the performance of affinity-based scheduling as a function of stream burstiness and source locality, iii) explore under IPS the impact of varying the number of independent protocol stacks, and iv) incorporate into our results the overhead of copying uncached packet data. We obtain our results following the research methodology developed in our earlier work, extending the developed infrastructure as necessary. Our results show that affinity-bas...

Abstract — The performance of the protocol stack implementation of an operating system can greatly impact the performance of networked applications that run on it. In this paper, we present a thorough measurement study and comparison of the network stack performance of the two popular Linux kernels: 2.4 and 2.6, with a special focus on their performance on SMP architectures. Our findings reveal that interrupt processing costs, device driver overheads, checksumming and buffer copying are dominant overheads of protocol processing. We find that although raw CPU costs are not very different between the two kernels, Linux 2.6 shows vastly improved scalability, attributed to better scheduling and kernel locking mechanisms. We also uncover an anomalous behaviour in which Linux 2.6 performance degrades when packet processing for a single connection is distributed over multiple processors. This, however, verifies the superiority of the “processor per connection ” model for parallel processing. I.

The dissertation of Douglas C. Schmidt is approved, and is acceptable in quality and form for publication on microfilm: