We propose a multi-object multi-camera framework for tracking large numbers of tightly-spaced objects that rapidly move in three dimensions. We formulate the problem of finding correspondences across multiple views as a multidimensional assignment problem and use a greedy randomized adaptive search procedure to solve this NPhard problem efficiently. To account for occlusions, we relax the one-to-one constraint that one measurement corresponds to one object and iteratively solve the relaxed assignment problem. After correspondences are established, object trajectories are estimated by stereoscopic reconstruction using an epipolar-neighborhood search. We embedded our method into a tracker-to-tracker multi-view fusion system that not only obtains the three-dimensional trajectories of closely-moving objects but also accurately settles track uncertainties that could not be resolved from single views due to occlusion. We conducted experiments to validate our greedy assignment procedure and our technique to recover from occlusions. We successfully track hundreds of flying bats and provide an analysis of their group behavior based on 150 reconstructed 3D trajectories. 1.

We developed two methods for tracking multiple objects using several camera views. The methods use the Multiple Hypothesis Tracking (MHT) framework to solve both the across-view data association problem (i.e., finding object correspondences across several views) and the across-time data association problem (i.e., the assignment of current object measurements to previously established object tracks). The “tracking-reconstruction method ” establishes two-dimensional (2D) objects tracks for each view and then reconstructs their three-dimensional (3D) motion trajectories. The “reconstruction-tracking method ” assembles 2D object measurements from all views, reconstructs 3D object positions, and then matches these 3D positions to previously established 3D object tracks to compute 3D motion trajectories. For both methods, we propose techniques for pruning the number of association hypotheses and for gathering track fragments. We tested and compared the performance of our methods on thermal infrared video of bats using several performance measures. Our analysis of video sequences with different levels of densities of flying bats reveals that the reconstruction-tracking method produces fewer track fragments than the trackingreconstruction method but creates more false positive 3D tracks. 1.

this paper we present our approach to 3D tracking of multiple SCRs observed by a moving stereoscopic system. This study was carried out in the context of a more-general research effort 36 toward developing a cognitive vision methodology to permit the interpretation of activities of people who are handling tools. Research and development are focused on the active observation and interpretation of the activities, on the extraction of the essential activities and their functional dependence, and on organizing the activities into their constituent behavior elements. The approach is active in the sense that the system seeks to obtain views that facilitate the interpretation of the activities observed. Therefore the ability to modify the viewpoint of observation of a certain activity is of utmost importance. Moreover, task and context knowledge is exploited as a means to constrain interpretation. Robust perception and interpretation of activities is the key to capturing the essential information that permits reproduction of task sequences from easy-to-understand representations. The system that we propose is able to track and report the 3D trajectories of all SCRs that are present in a viewed scene. The proposed method for detecting SCRs has several attractive properties. A skincolor representation is learned through an off-line procedure. A new technique is proposed that eliminates much of the burden involved in generating training data. Moreover, the method adapts the skin-color model based on the recent history of tracked SCRs. Thus, without the need for complex models, the proposed approach is able to detect SCRs robustly and efficiently, even in conditions of changing illumination. The system employs a moving stereoscopic rig with cameras that have independent vergence...

I want to especially thank my advisor, Prof. Margrit Betke for her mentorship in every aspect of my research life. Margrit has been very patient to instruct me with every piece of my work in the past six years, providing extremely professional guidance and kind encouragement to help shape my research career. I would also like to acknowledge all