The boundary between application and system is becoming increasingly permeable. Extensible applications, such as web browsers, database systems, and operating systems, demonstrate the value of allowing end-users to extend and modify the behavior of what was formerly considered to be a static, inviolate system. Unfortunately, flexibility often comes with a cost: systems unprotected from misbehaved end-user extensions are fragile and prone to instability. Object-oriented programming models are a good fit for the development of this kind of system. An extensions can be designed as a refinement of an existing class, and loaded into a running system. In our model, when code is downloaded into the system, it is used to replace a virtual function on an existing C++ object. Because our tool is source-language neutral, it can be used to build safe extensible systems written in other languages as well. There are three methods commonly used to make end-user extensions safe: restrict the extension language (e.g., Java), interpret the extension language (e.g., Tcl), or combine run-time checks with a trusted environment. The third technique is the one discussed here; it offers the twin benefits of the flexibility to implement extensions in an unsafe language, such as C++, and the performance of compiled code. MiSFIT, the Minimal i386 Software Fault Isolation Tool, can be used as the central component of a tool set for building safe extensible systems in C++. MiSFIT transforms C++ code, compiled by g++, into safe binary code. Combined with a runtime support library, the overhead of MiSFIT is an order of magnitude lower than the overhead of interpreted Java, and permits safe extensible systems to be written in C++. 1

When designing an extensible operating system, a developer must ensure that the operating system is protected from misbehaved extensions. Two kinds of protection are needed: first, extensions should not violate the operating system’s interface, and second, extensions should not be able to leave the operating system in an inconsistent state. The major research contributions of this thesis include: The design and evaluation of MiSFIT, a software fault isolation tool for the x86 architecture that ensures that extensions do not violate the operating system’s interface and incurs minimal overhead. The design and evaluation of VINO Lightweight Transactions, a low-overhead mechanism that allows the kernel to maintain its consistency in the face of ill-behaved extensions. Experiments that show the end-to-end overhead of MiSFIT and VLT protection is low, on the order of 1-2%, and the net performance gain possible from using application-specific extensions is significant, in some cases more than 20%. A cost-benefit framework for comparing extension technologies and an evaluation comparing

The boundary between application and system is becoming increasingly permeable. Extensible database systems, operating systems, and applications, such as web browsers, are demonstrating the value of allowing end-users to extend and modify the behavior of what formerly was considered to be a static, inviolate system. Unfortunately, flexibility often comes with a cost: systems unprotected from misbehaved end-user extensions are fragile and prone to instability. There are three common methods for making enduser extensions safe: restrict the extension language, interpret the extension language, or add run-time checks to binary code that ensure the safety of an otherwise unsafe program. The third technique, software fault isolation, offers the twin benefits of the performance of compiled code and the flexibility to choose an unsafe language, such as C or C++. MiSFIT, a tool for software fault isolation of x86 code, transforms unsafe C or C++ into safe binary code. The performance overhead o...

1:20 am Page 2 of 3 [FM96] S. J. Fritzinger and M. Mueller. Java security. http://www.javasoft.com/security/whitepaper.txt, 1996. [HCG + 98] C. Hawblitzel, C. Chang, G. Gzajkowski, D. Hu, and T. von Eicken. Implementing multiple protection domains in Java. In Proceedings of the USENIX 1998 Annual Technical Conference, New Orleans, La., June 1998. [HEVR97] G. Heiser, K. Elphinstone, J. Vochteloo, and S. Russell. Implementation and performance of the Mungi single-address-space operating system. Technical Report UNSW-CSE-TR-9704, The University of New South Wales, Sydney, Australia, June 1997. [HHL + 97] H. Hartic, M. Hohmuth, J. Liedtke, S. Schonberg, and J. Wolter. The performance of -kernel- based systems. In 16th ACM Symposium on Operating Systems Principles (SOSP '97), SaintMalo, France, October 1997. [Hoa78] C. A. R. Hoare. Communicating sequential processes. Communications o