Although machine learning is becoming commonly used in today’s software, there has been little research into how end users might interact with machine learning systems, beyond communicating simple "right/wrong " judgments. If the users themselves could work hand-in-hand with machine learning systems, the users ’ understanding and trust of the system could improve and the accuracy of learning systems could be improved as well. We conducted three experiments to understand the potential for rich interactions between users and machine learning systems. The first experiment was a think-aloud study that investigated users ’ willingness to interact with machine learning reasoning, and what kinds of feedback users might give to machine learning systems. We then investigated the viability of introducing such feedback into machine learning systems, specifically, how to incorporate some of these types of user feedback into machine learning systems, and what their impact was on the accuracy of the system. Taken together, the results of our experiments show that supporting rich interactions between users and machine learning systems is feasible for both user and machine. This shows the potential of rich humancomputer collaboration via on-the-spot interactions as a promising direction for machine learning systems and users to collaboratively share intelligence.

Mixed-initiative clustering is a task where a user and a machine work collaboratively to analyze a large set of documents. We hypothesize that a user and a machine can both learn better clustering models through enriched communication and interactive learning from each other. The first contribution of this thesis is providing a framework of mixedinitiative clustering. The framework consists of machine learning and teaching phases, and user learning and teaching phases connected in an interactive loop which allows bi-directional communication. The bi-directional communication languages define types of information exchanged in an interface. Coordination between the two communication languages and the adaptation capability of the machine’s clustering model is the key to building a mixed-initiative clustering system. The second contribution comes from successfully building several systems using our proposed framework. Two systems are built with incrementally enriched communication languages – one enables user feedback on features for

Abstract. In most major search engines, the interface for image search is the same as traditional Web search: a keyword query followed by a paginated, ranked list of results. Although many image search innovations have appeared in both the literature and on the Web, few have seen widespread use in practice. In this work, we explore the differences between image and general Web search to better support users ‟ needs. First, we describe some unique characteristics of image search derived through informal interviews with researchers, designers, and managers responsible for building and deploying a major Web search engine. Then, we present results from a large scale analysis of image and Web search logs showing the differences in user behaviour. Grounded in these observations, we present design recommendations for an image search engine supportive of the unique experience of image search. We iterate on a number of designs, and describe a functional prototype that we built.

We present ReGroup, a novel end-user interactive machine learning system for helping people create custom, on-demand groups in online social networks. As a person adds members to a group, ReGroup iteratively learns a probabilistic model of group membership specific to that group. ReGroup then uses its currently learned model to suggest additional members and group characteristics for filtering. Our evaluation shows that ReGroup is effective for helping people create large and varied groups, whereas traditional methods (searching by name or selecting from an alphabetical list) are better suited for small groups whose members can be easily recalled by name. By facilitating on-demand group creation, ReGroup can enable in-context sharing and potentially encourage better online privacy practices. In addition, applying interactive machine learning to social network group creation introduces several challenges for designing effective end-user interaction with machine learning. We identify these challenges and discuss how we address them in ReGroup. Author Keywords Interactive machine learning, social network group creation, access control lists, example and feature-based interaction.

End-user interactive concept learning is a technique for interacting with large unstructured datasets, requiring insights from both human-computer interaction and machine learning. This note re-examines an assumption implicit in prior interactive machine learning research, that interaction should focus on the question “what class is this object?”. We broaden interaction to include examination of multiple potential models while training a machine learning system. We evaluate this approach and find that people naturally adopt revision in the interactive machine learning process and that this improves the quality of their resulting models for difficult concepts.

interface for generating automatic suggestions (a) and three example-based interaction techniques for iteratively refining a collection (b–d). To create collections, like music playlists from personal media libraries, users today typically do one of two things. They either manually select items one-by-one, which can be time consuming, or they use an example-based recommendation system to automatically generate a collection. While such automatic engines are convenient, they offer the user limited control over how items are selected. Based on prior research and our own observations of existing practices, we propose a semi-automatic interface for creating collections that combines automatic suggestions with manual refinement tools. Our system includes a keyword query interface for specifying high-level collection preferences (e.g., “some rock, no

We present an interactive learning method that enables a user to iteratively refine a regression model. The user examines the output of the model, visualized as the vertical axis of a 2D scatterplot, and provides corrections by repositioning individual data instances to the correct output level. Each repositioned data instance acts as a control point for altering the learned model, using the geometry underlying the data. We capture the underlying structure of the data as a manifold, on which we compute a set of basis functions as the foundation for learning. Our results show that manifold-based interactive learning improves performance monotonically with each correction, outperforming alternative approaches.

End-user interactive concept learning is a technique for interacting with large unstructured datasets, requiring insights from both human-computer interaction and machine learning. This note re-examines an assumption implicit in prior interactive machine learning research, that interaction should focus on the question “what class is this object?”. We broaden interaction to include examination of multiple potential models while training a machine learning system. We evaluate this approach and find that people naturally adopt revision in the interactive machine learning process and that this improves the quality of their resulting models for difficult concepts.

The current information explosion fundamentally changes how people live and work with computing: vast numbers of documents and images are available on the Web; ubiquitous sensing enables near-continuous tracking and monitoring of people and objects; and inexpensive storage allows people to keep near-unlimited personal data and sensing archives. One strategy to enable effective access to and interaction with such large unstructured datasets is to support example-based iterative end-user training of machine learning systems to identify relevant concepts. These concepts can then be used specify desired manipulations. In the context of the CueFlik system for re-ranking Web image search results according to their visual characteristics, we have been examining general questions surrounding the design of end-user interactive machine learning. By iteratively providing examples and inspecting the resulting model, end-users can train machine learning systems to

We address the problem of incorporating user preference in automatic image enhancement. Unlike generic tools for automatically enhancing images, we seek to develop methods that can first observe user preferences on a training set, and then learn a model of these preferences to personalize enhancement of unseen images. The challenge of designing such system lies at intersection of computer vision, learning, and usability; we use techniques such as active sensor selection and distance metric learning in order to solve the problem. The experimental evaluation based on user studies indicates that different users do have different preferences in image enhancement, which suggests that personalization can further help improve the subjective quality of generic image enhancements. 1.