We describe a new approach to the detection and classification of scene breaks in video sequences. Our method can detect and classify a variety of scene breaks, including cuts, fades, dissolves and wipes, even in sequences involving signi cant motion. We detect the appearance of intensity edges that are distant from edges in the previous frame. A global motion computation is used to handle camera or object motion. The algorithm we propose withstands JPEG and MPEG artifacts, even at very high compression rates. Experimental evidence demonstrates that our method can detect and classify scene breaks that are difficult to detect with previous approaches. An initial implementation runs at approximately 2 frames per second on a Sun workstation.

Most robotic grasping tasks assume a stationary or fixed object. In this paper, we explore the requirements for tracking and grasping a moving object. The focus of our work is to achieve a high level of interaction between a real-time vision system capable of tracking moving objects in 3-D and a robot arm equipped with a dexterous hand that can be used pick up a moving object. We are interested in exploring the interplay of hand-eye coordination for dynamic grasping tasks such as grasping of parts on a moving conveyor system, assembly of articulated parts or for grasping from a mobile robotic system. Coordination between an organism's sensing modalities and motor control system is a hallmark of intelligent behavior, and we are pursuing the goal of building an integrated sensing and actuation system that can operate in dynamic as opposed to static environments. The system we have built addresses three distinct problems in robotic hand-eye coordination for grasping moving objects: fast computation of 3-d motion parameters from vision, predictive control of moving robotic arm to track a moving oblect, and grasp planning. The system is able to operate at approximately human arm movement rates, and we present experimenatl result in which a moving model train is tracked, stably grasped, and picked up by the system. The algorithms we have developed that relate sensing to actuation are quite general and applicable to a variety of complex robotic tasks that require visual feedback for arm and hand control.

This paper describes a new real-time tracking algorithm in conjunction with a predictive filter to allow real-time visual servoing of a robotic arm that is following a moving object. The system consists of two calibrated (but unregistered) cameras that provide images to a real-time, pipelined-parallel optic-flow algorithm that can robustly compute optic-flow and calculate the 3-D position of a moving object at approximately 5 Hz rates. These 3-D positions of the moving object serve as input to a predictive kinematic control algorithm that uses an a - b - g filter to update the position of a robotic arm tracking the moving object. Experimental results are presented for the tracking of a moving model train in a variety of different trajectories. 1. INTRODUCTION Tracking the three-dimensional movement of objects by a vision system in real-time is an important problem. It has been addressed by researchers in a number of different fields including target tracking, surveillance, automated ...

Currently two major limitations to applying vision in real tasks are robustness in realworld, uncontrolled environments, and the computational resources required for real-time operation. In particular, many current robotic visual motion detection algorithms (optical flow) are not suited for practical applications such as segmentation and structure-frommotion because they either require highly specialized hardware or up to several minutes on a scientific workstation. In addition, many such algorithms depend on the computation of first and in some cases higher numerical derivatives, which are notoriously sensitive to noise. In fact the current trend in optical flow research is to stress accuracy under ideal conditions and not to consider computational resource requirements or resistance to noise, which are essential for real-time robotics. As a result robotic vision researchers are frustrated by an inability to obtain reliable optical flow estimates in real-world conditions, and practica...

In this paper we describe an active binocular tracking system integrating peripheral motion detection. The system is made up of a binocular active system used to track the objects and a fixed camera providing wide angle images of the environment. The system can cope with changes in lighting conditions by adjusting aperture and focus. Binocular flow enables tracking of non-rigid objects even when partial occlusion occurs. 1 Introduction Surveillance applications are one of the most relevant applications of computer vision. The scenarios of surveillance applications are also extremely varied [12, 11, 13, 17, 18, 15]. Some applications are related to traffic monitoring and surveillance [13, 14], others are related to surveillance in large regions for human activity [16], and there are also applications (related to security) that may imply behavior modelling and analysis [21, 22, 23, 24, 25]. For security applications in man-made environments video images are the most important type of d...

An active vision system has to enable the implementation of reactive visual processes and of elementary visual behaviors in real time. Therefore the control architecture is extremely important. In this paper we discuss a number of issues related with the implementation of a real-time control architecture and describe the architecture we are using with our camera heads. Even though in most applications a fully calibrated system is not required, we also describe a methodology for calibrating the camera head, taking advantage of its degrees of freedom. These calibration parameters are used to evaluate the performance of the system. Another important issue of the operation of active vision binocular heads is their integration into more complex robotic systems. We claim that higher levels of autonomy and integration can be obtained by designing the system architecture based on the concept of purposive behavior. At the lower levels we consider vision as a sensor and integrate it in control s...

The evolution of an image sequence obtained by a real camera from a real scene can be conceptually separated into two parts: 1) motion of the camera and 2) motion of the objects in a scene. Most existing motion estimation algorithms use the block matching algorithm (BMA) to model both the camera motion and local motion due to the objects. In doing so, successive frames are divided into small blocks and the movement of each block is approximately modeled by a translation, thus resulting in one motion vector per block. In this paper, we propose two classes of algorithms for modeling camera motion in video sequences captured by a camera. The first class can be applied in situations where there is no camera translation and the motion of camera can be adequately modeled by zoom, pan, and rotation parameters. The second class is more general in that it can be applied to situations where the camera is undergoing a translational motion, as well as a rotation and zoom and pan. This class uses seven parameters to describe the motion of the camera and requires the depth map to be known at the receiver. The salient feature of both of our algorithms is that the camera motion is estimated using binary matching of the edges in successive frames. In doing so, we show that unlike local motion estimation, edge matching can be sufficient in estimating camera motion parameters. Finally, we compare the rate distortion characteristics of our algorithms with that of the BMA and show that we can achieve similar performance characteristics as BMA, with reduced computational complexity.

A measure of motion salience is proposed for surveillance applications. In this context, salient motion is taken as motion that is likely to result from a typical surveillance target #e.g., a person or vehicle traveling with a sense of direction through a scene# as opposed to other distracting motions #e.g., the scintillation of specularities on water, the oscillation of trees in the wind#. The measure that is proposed makes use of spatiotemporal gradient #lters to characterize the extent to which a single coherent motion dominates local regions in space-time. Empirical results show that the measure is capable of making the desired distinctions. The signi#cance of the approach lies in its ability to provide information about motion salience based on relatively simple, early vision operations. Such information can provide vital input to subsequent processing in attempting to distinguish targets of interest from mere distractors. 1 Introduction Visual motion can be an important source ...

In this paper we describe the implementation of real-time binocular gaze control. This implementation is performed by using a complex binocular active vision system. In addition real-time vergence is also implemented. Vergence and gaze control are implemented based on binocular optical flow. Vergence control is implemented based on optical flow disparity, using the horopter as a reference. A kalman filter is used to filter out the signals and compensate for the delays of the system. 1 Introduction The full recovery of the 3D structure (shape, location and other properties) of the world has proved to be an extremely difficult task. In addition most of the mathematical problems involved in such 3D reconstruction are ill-posed. For the vast majority of vision problems all 3D information is unnecessary. Biological vision systems prove that Nature tailored these systems to the particular needs of each animal. Instead of trying to find general solutions for the vision modules we can consid...

This paper presents a method for doing motion segmentation for autonomous vehicles which drive on planar surfaces. There are two distinct types of independent motion that may occur within an image sequence taken from a moving vehicle. The first generic type of independent motion is when the projected motion of points on the independent object violate the epipolar constraint. The second case is where the epipolar constraint is not violated. This paper demonstrates that it is possible to detect this second type of independent motion by looking for progressive dis-occlusion of the road. A novel collision prediction method is also given. The method predicts the projection of a corridor down which the AGV will travel. This prediction may be used for time to contact collision prediction and the corridor width embodies an estimate of the vehicles size.