Twenty years after the general adoption of overlapping windows and the desktop metaphor, modern window systems differ mainly in minor details such as window decorations or mouse and keyboard bindings. While a number of innovative window management techniques have been proposed, few of them have been evaluated and fewer have made their way into real systems. We believe that one reason for this is that most of the proposed techniques have been designed using a low fidelity approach and were never made properly available. In this paper, we present Metisse, a fully functional window system specifically created to facilitate the design, the implementation and the evaluation of innovative window management techniques. We describe the architecture of the system, some of its implementation details and present several examples that illustrate its potential.

Current chasms between applications implemented with different user interface toolkits make it difficult to implement and explore potentially important interaction techniques in new and existing applications, limiting the progress and impact of human-computer interaction research. We examine an approach based in the single most common characteristic of all graphical user interface toolkits, that they ultimately paint pixels to a display. We present Prefab, a system for implementing advanced behaviors through the reverse engineering of the pixels in graphical interfaces. Informed by how user interface toolkits paint interfaces, Prefab features a separation of the modeling of widget layout from the recognition of widget appearance. We validate Prefab in implementations of three applications: target-aware pointing techniques, Phosphor transitions, and Side Views parameter spectrums. Working only from pixels, we demonstrate a single implementation of these enhancements in complex existing applications created in different user interface toolkits running on different windowing systems.

Copy-and-paste, one of the fundamental operations of modern user interfaces, can be performed through various means (e.g. using the keyboard, mouse-based direct manipulation or menus). When users copy and paste between two different windows, the process is complicated by window management tasks. In this paper, we propose two new window management techniques to facilitate these tasks in the particular case of partially overlapping windows. We describe an experiment comparing four commonly used copy-andpaste techniques under four window management conditions – non-overlapping windows, partially overlapping windows, and partially overlapping ones with one of our two window management techniques. Results show that our new window management techniques significantly reduce task completion time for all copy-and-paste techniques. They also show that X Window copy-and-paste is faster than the other three techniques under all four window management conditions.

We present a new server-side architecture that enables rapid prototyping and deployment of mobile web applications created from existing web sites. Key to this architecture is a remote control metaphor in which the mobile device controls a fully functional browser that is embedded within a proxy server. Content is clipped from the proxy browser, transformed if necessary, and then sent to the mobile device as a typical web page. Users ’ interactions with that content on the mobile device control the next steps of the proxy browser. We have found this approach to work well for creating mobile sites from a variety of existing sites, including those that use dynamic HTML and AJAX technologies. We have conducted a small user study to evaluate our model and API with experienced web programmers. ACM Classification: H.5.4 [Hypertext/Hypermedia]:

Abstract. Plasticity of User Interfaces (UIs) refers to the ability of UIs to withstand variations of context of use (<User, Platform, Environment>) while preserving usability. This paper presents COMET, a software architecture style for building task-based plastic interactors. COMET bridges the gap between two main approaches in plasticity: model-driven engineering and interactors toolkits. Interactors that are compliant to the COMET style are called COMETs. These COMETs are multi-rendering multi-technological interactors (WIMP and post-WIMP, Web and non Web as well as vocal). COMETs are extensible and controllable by the user (up until now the designer, in the future the end-user). The COMET architecture and the use of COMETs are illustrated on an executable prototype: a slide viewer called CamNote++.

Figure 1: Entering the UIMarks mode (A → B), selecting the top mark (C), activating this mark, which double-clicks and sends the cursor back to the entering point (C → D). In this note we present UIMarks, a novel system that lets users specify on-screen targets and associated actions by means of graphical marks. UIMarks supplements traditional pointing by providing users with an alternative mode in which they can quickly create, select, configure, activate and delete marks. Actions associated to these marks can range from basic point-and-click interactions to the execution of complex action sequences. UIMarks has been implemented on two different platforms, Metisse and OS X. ACM Classification: H.5.2 [Information interfaces and presentation]: User interfaces- Graphical user interfaces.

Personalized graphical user interfaces have the potential to reduce visual complexity and improve interaction efficiency by tailoring elements such as menus and toolbars to better suit an individual user's needs. When an interface is personalized to make useful features more accessible for a user's current task, however, there may be a negative impact on the user's awareness of the full set of available features, making future tasks more difficult. To assess this tradeoff we introduce awareness as an evaluation metric to be used in conjunction with performance. We then discuss three studies we have conducted, which show that personalized interfaces trade off awareness of unused features for performance gains on core tasks. The first two studies, previously published and presented only in summary, demonstrate this tradeoff by measuring awareness using a recognition test of unused features in the interface. The studies also evaluated two different types of personalized interfaces: a layered interfaces approach and an adaptive split menu approach. The third study, presented in full, focuses on adaptive split menus and extends results from the first two studies to show that different levels of awareness also correspond to an impact on performance when users are asked to complete new tasks. Based on all three studies and a survey of related work, we outline a design space of personalized interfaces and present several factors that could affect the tradeoff between core task performance and awareness. Finally, we provide a set of design implications that should be considered for personalized interfaces.

and parts thereof, including all revisions or versions and future editions thereof and in any medium, such as

We introduce GEKA, a graphically enhanced keyboard accelerator method that provides the advantages of a traditional command line interface within a GUI environment, thus avoiding the “Fitts-induced bottleneck ” of pointer movement that is characteristic of most WIMP methods. Our design rationale and prototype development were derived from a small formative user study, which suggested that advanced users would like alternatives to WIMP methods in GUIs. The results of a controlled experiment show that GEKA performs well, is faster than menu selection, and is strongly preferred over all mouse-based WIMP methods.

Pixel-based methods emerge to be a new and promising way for developing new interaction techniques on top of existing user interfaces. However, in order to maintain platform independence, other available low-level information about GUI widgets, such as accessibility metadata, was neglected intentionally. In this paper, we present a hybrid framework, PAX, which correlates the visual representation of user interfaces (i.e. the pixels) and their internal hierarchical metadata (i.e. the content, role, and value). We identify challenges to build such a framework. We also develop and evaluate two new algorithms for detecting text at arbitrary places on the screen, and for segmenting a text image into individual word blobs. Finally, we validate our framework in implementations of three applications. We enhance an existing pixel-based system, Sikuli Script, and preserve the readability of its script code at the same time. Further, we create two novel applications, Screen Search and Screen Copy, to demonstrate how PAX can be applied to developing desktop-level interactive systems. ACM Classification: H5.2. Information interfaces and presentation: User Interfaces.