Operating systems must be flexible in their support for security policies, providing sufficient mechanisms for supporting the wide variety of real-world security policies. Such flexibility requires controlling the propagation of access rights, enforcing fine-grained access rights and supporting the revocation of previously granted access rights. Previous systems are lacking in at least one of these areas. In this paper we present an operating system security architecture that solves these problems. Control over propagation is provided by ensuring that the security policy is consulted for every security decision. This control is achieved without significant performance degradation through the use of a security decision caching mechanism that ensures a consistent view of policy decisions. Both fine-grained access rights and revocation support are provided by mechanisms that are directly integrated into the service-providing components of the system. The architecture is described through its prototype implementation in the Flask microkernelbased operating system, and the policy flexibility of the prototype is evaluated. We present initial evidence that the architecture’s impact on both performance and code complexity is modest. Moreover, our architecture is applicable to many other types of operating systems and environments. 1

Traditional user-oriented access control models such as Mandatory Access Control (MAC) and Discretionary Access Control (DAC) cannot differentiate between processes acting on behalf of users and those behaving maliciously. Consequently, these models are limited in their ability to protect users from the threats posed by vulnerabilities and malicious software as all code executes with full access to all of a user's permissions. Application-oriented schemes can further restrict applications thereby limiting the damage from malicious code. However, existing application-oriented access controls construct policy using complex and inflexible rules which are difficult to administer and do not scale well to confine the large number of feature-rich applications found on modern systems. Here a new model, Functionality-Based Application Confinement (FBAC), is presented which confines applications based on policy abstractions that can flexibly represent the functional requirements of applications. FBAC policies are parameterised allowing them to be easily adapted to the needs of individual applications. Policies are also hierarchical, improving scalability and reusability while conveniently abstracting policy detail where appropriate. Furthermore the layered nature of policies provides defence in depth allowing policies from both the user and administrator to provide both discretionary and mandatory security. An implementation FBAC-LSM and its architecture are also introduced.