Abstract. We study activity recognition using 104 hours of annotated data collected from a person living in an instrumented home. The home contained over 900 sensor inputs, including wired reed switches, current and water flow inputs, object and person motion detectors, and RFID tags. Our aim was to compare different sensor modalities on data that approached “real world ” conditions, where the subject and annotator were unaffiliated with the authors. We found that 10 infra-red motion detectors outperformed the other sensors on many of the activities studied, especially those that were typically performed in the same location. However, several activities, in particular “eating ” and “reading ” were difficult to detect, and we lacked data to study many fine-grained activities. We characterize a number of issues important for designing activity detection systems that may not have been as evident in prior work when data was collected under more controlled conditions. 1

Sensor networks hold the promise of truly intelligent buildings: buildings that adapt to the behavior of their occupants to improve productivity, efficiency, safety, and security. To be practical, such a network must be economical to manufacture, install and maintain. Similarly, the methodology must be efficient and must scale well to very large spaces. Finally, be be widely acceptable, it must be inherently privacy-sensitive. We propose to address these requirements by employing networks of passive infrared (PIR) motion detectors. PIR sensors are inexpensive, reliable, and require very little bandwidth. They also protect privacy since they are neither capable of directly identifying individuals nor of capturing identifiable imagery or audio. However, with an appropriate analysis methodology, we show that they are capable of providing useful contextual information. The methodology we propose supports scalability by adopting a hierarchical framework that splits computation into localized, distributed tasks. To support our methodology we provide theoretical justification for the method that grounds it in the action recognition literature. We also present quantitative results on a dataset that we have recorded from a 400 square meter wing of our laboratory. Specifically, we report quantitative results that show better than 90 % recognition performance for low-level activities such as walking, loitering, and turning. We also present experimental results for mid-level activities such as visiting and meeting.

This is a literature survey of computational location privacy, meaning computation-based privacy mechanisms that treat location data as geometric information. This definition includes privacy-preserving algorithms like anonymity and obfuscation as well as privacy-breaking algorithms that exploit the geometric nature of the data. The survey omits non-computational techniques like manually inspecting geotagged photos, and it omits techniques like encryption or access control that treat location data as general symbols. The paper reviews studies of peoples’ attitudes about location privacy, computational threats on leaked location data, and computational countermeasures for mitigating these threats.

The home deployment of sensor-based systems offers many opportunities, particularly in the area of using sensor-based systems to support aging in place by monitoring an elder’s activities of daily living. But existing approaches to home activity recognition are typically expensive, difficult to install, or intrude into the living space. This paper considers the feasibility of a new approach that “reaches into the home ” via the existing infrastructure. Specifically, we deploy a small number of low-cost sensors at critical locations in a home’s water distribution infrastructure. Based on water usage patterns, we can then infer activities in the home. To examine the feasibility of this approach, we deployed real sensors into a real home for six weeks. Among other findings, we show that a model built on microphone-based sensors that are placed away from systematic noise sources can identify 100 % of clothes washer usage, 95 % of dishwasher usage, 94 % of showers, 88 % of toilet flushes, 73 % of bathroom sink activity lasting ten seconds or longer, and 81 % of kitchen sink activity lasting ten seconds or longer. While there are clear limits to what activities can be detected when analyzing water usage, our new approach represents a sweet spot in the tradeoff between what information is collected at what cost.

Abstract. In this paper, we introduce MITes, a flexible kit of wireless sensing devices for pervasive computing research in natural settings. The sensors have been optimized for ease of use, ease of installation, affordability, and robustness to environmental conditions in complex spaces such as homes. The kit includes six environmental sensors: movement, movement tuned for object-usage-detection, light, temperature, proximity, and current sensing in electric appliances. The kit also includes five wearable sensors: onbody acceleration, heart rate, ultra-violet radiation exposure, RFID reader wristband, and location beacons. The sensors can be used simultaneously with a single receiver in the same environment. This paper describes our design goals and results of the evaluation of some of the sensors and their performance characteristics. Also described is how the kit is being used for acquisition of data in non-laboratory settings where real-time multi-modal sensor information is acquired simultaneously from several sensors worn on the body and up to several hundred sensors distributed in an environment. 1

By analyzing the similarities between bit streams coming from a network of motion detectors, we can recover the network geometry and discover structure in the human behavior being observed. This means that a low-cost network of sensors can provide powerful contextual information to building systems: improving the efficiency of elevators, lighting, heating, and cooling; enhancing safety and security; and opening up new opportunities for human-centered information systems. This paper will show how signal similarity can be used to calibrate a sensor network to accuracies below the resolution of the individual sensors. This is done by analyzing the similarity structures in the unconstrained movement of people in the observed space. We will also present our efficient behavior-learning algorithm that yields 90 % correct behavior-detection in data from a sensor network comprised of motion detectors by employing similarity-based clustering to automatically decompose complex activities into detectable sub-classes.

Abstract. We have developed an approach for whole-house gross movement and room transition detection through sensing at only one point in the home. We consider this system to be one member of an important new class of human activity monitoring approaches based on what we call infrastructure mediated sensing, or "home bus snooping. " Our solution leverages the existing ductwork infrastructure of central heating, ventilation, and air conditioning (HVAC) systems found in many homes. Disruptions in airflow, caused by human interroom movement, result in static pressure changes in the HVAC air handler unit. This is particularly apparent for room-to-room transitions and door open/close events involving full or partial blockage of doorways and thresholds. We detect and record this pressure variation from sensors mounted on the air filter and classify where certain movement events are occurring in the house, such as an adult walking through a particular doorway or the opening and closing of a particular door. In contrast to more complex distributed sensing approaches for motion detection in the home, our method requires the installation of only a single sensing unit (i.e., an instrumented air filter) connected to an embedded or personal computer that performs the classification function. Preliminary results show we can classify unique transition events with up to 75-80 % accuracy. 1

Looking into the future of residential and office building Mitsubishi Electric Research Labs (MERL) has been collecting motion sensor data from a network of over 200 sensors for a year. The data is the residual traces of year in the life of a research laboratory. It contains interesting spatio-temporal structure ranging all the way from the seconds of individuals walking down hallways, the minutes in the lobbies chatting with colleagues, the hours of dozens of people attending talks and meetings, the days and weeks that drive the patterns of life, to the months and seasons with their ebb and flow of visiting employees. This document describes that dataset, which contains well over 30 million raw motion records, spanning a calendar year and two floors of our research laboratory, as well as calendar, weather, and some intermediate analytic results.

In traditional surveillance systems tracking of objects is achieved by means of image and video processing. The disadvantages of such surveillance systems is that if an object needs to be tracked- it has to be observed by a video camera. However, geometries of indoor spaces typically require a large number of video cameras to provide the coverage necessary for robust operation of video-based tracking algorithms. Increased number of video streams increases the computational burden on the surveillance system in order to obtain robust tracking results. In this paper we present an approach to tracking in mixed modality systems, with a variety of sensors. The system described here includes over 200 motion sensors as well as 6 moving cameras. We track individuals in the entire space and across cameras using contextual information available from the motion sensors. Motion sensors allow us to almost instantaneously find plausible tracks in a very large volume of data, ranging in months, which for traditional video search approaches could be virtually impossible. We describe a method that allows us to evaluate when the tracking system is unreliable and present the data to a human operator for disambiguation.

Abstract. Pervasive sensors in the home have a variety of applications including energy minimization, activity monitoring for elders, and tutors for household tasks such as cooking. Many of the common sensors today are binary, e.g. IR motion sensors, door close sensors, and floor pressure pads. Predicting user behavior is one of the key enablers for applications. While we consider smart home data here, the general problem is one of predicting discrete human actions. Drawing on Activity Theory, the Language-as-Action principle, and Speech understanding research, we argue that smoothed n-grams are very appropriate for this task. We built such a model and applied it to data gathered from 3 smart home installations. The data showed a classic Zipf or power-law distribution, similar to speech and language. We found that the predictive accuracy of the n-gram model ranges from 51 % to 39%, which is significantly above the baseline for the deployments of 16, 76 and 70 sensors. While we cannot directly compare this result with other work (lack of shared data), by examination of high entropy zones in the datasets (e.g. the kitchen triangle) we argue that accuracies around 50 % are best possible for this task. 1