This paper provides a tutorial introduction to visual servo control of robotic manipulators. Since the topic spans many disciplines our goal is limited to providing a basic conceptual framework. We begin by reviewing the prerequisite topics from robotics and computer vision, including a brief review of coordinate transformations, velocity representation, and a description of the geometric aspects of the image formation process. We then present a taxonomy of visual servo control systems. The two major classes of systems, position-based and image-based systems, are then discussed. Since any visual servo system must be capable of tracking image features in a sequence of images, we include an overview of feature-based and correlation-based methods for tracking. We conclude the tutorial with a number of observations on the current directions of the research field of visual servo control. 1 Introduction Today there are over 800,000 robots in the world, mostly working in factory environment...

Intelligent agents perform multiple concurrent tasks requiring both knowledge-based reasoning and interaction with dynamic entities in the environment, under real-time constraints. Because an agent's opportunities to perceive, reason about, and act upon the environment typically exceed its computational resources, it must determine which operations to perform and when to perform them so as to achieve its most important objectives in a timely manner. Accordingly, we view the problem of real-time performance as a problem in intelligent real-time control. We propose and define several important control requirements and present an agent architecture that is designed to address those requirements. The proposed architecture is a blackboard architecture, whose key features include: distribution of perception, action, and cognition among parallel processes, limited-capacity I/O buffers with best-first retrieval and worst-first overflow, dynamic control planning, dynamic focus of attention, and...

Good Old Fashioned Artificial Intelligence and Robotics (GOFAIR) relies on a set of restrictive Omniscient Fortune Teller Assumptions about the agent, the world and their relationship. The emerging Situated Agent paradigm is challenging GOFAIR by grounding the agent in space and time, relaxing some of those assumptions, proposing new architectures and integrating perception, reasoning and action in behavioral modules. GOFAIR is typically forced to adopt a hybrid architecture for integrating signal-based and symbol-based approaches because of the inherent mismatch between the corresponding on-line and off-line computational models. It is argued that Situated Agents should be designed using a unitary on-line computational model. The Constraint Net model of Zhang and Mackworth satisfies that requirement. Two systems for situated perception built in our laboratory are described to illustrate the new approach: one for visual monitoring of a robot's arm, the other for real-time visual control of multiple robots competing and cooperating in a dynamic world.

A real-time AI system in the real world needs to monitor an immense volume of data. To do this, the system must filter out much of the incoming data. However, it must remain responsive to important or unexpected events in the data. This paper describes some simple approaches to data management, shows how they can fail to be both adequately selective and responsive, and presents an approach that improves on the simple approaches by making use of information about the system's resources and ongoing tasks. The new approach has been applied in a system for monitoring patients in a surgical intensive-care unit. 1

Our mobile robot, Spinoza, embodies a sophisticated real-time vision system for control of a mobile robot in a dynamic environment. The complexity of our robot architecture arises from the wide variety of tasks that need to be performed and the resulting chal-lenge of coordinating multiple distributed, concurrent processes on a diverse range of processor architec-tures including Transputers, digital signal processors, and a workstation host. The system handles sens-ing, reasoning, and action components of a robot dis-tributed over these architectures, and responds to un-predictable events in an unknown dynamic environ-ment. Spinoza relies heavily on its capability to per-form real-time vision processing in order to perform task such as mapping, navigation, exploration, track-ing, and simple manipulation. 1

Abstract — An image-based visual servo control is presented for an Unmanned aerial vehicle (UAV) capable of stationary or quasi-stationary flight. The proposed control design addresses visual servo of ‘eye-in-hand ’ type systems. The control of the position and orientation dynamics are decoupled using a visual error based on a spherical centroid data, along with estimation of the gravitational inertial direction. The error used compensates for the poor conditioning of the Jacobian matrix seen in earlier work in this area by introducing a non-homogeneous gain term adapted to the visual sensitivity of the error measurements. A nonlinear controller is derived for the full dynamics of the system. Experimental results on an experimental UAV known as an X4-flyer made by the French Atomic Energy Commission (CEA) demonstrate the robustness and performance of the proposed control strategy. I.

The task and design requirements for a vision system for manipulator position sensing in a telerobotic system are described. Model-based analysis-by-synthesis techniques offer generally applicable methods with the potential to meet the system's requirement for accurate, fast and reliable results. Edge-based chamfer matching allows efficient computation of a measure, E, of the local difference between the real image and a synthetic image generated from arm and camera models. Gradient descent techniques are used to minimise E by adjusting joint angles. The dependence of each link position on the position of the link preceding it allows the search to be broken down into lower dimensional problems. Intensive exploitation of geometric constraints on the possible position and orientation of manipulator components results in a correct and efficient solution to the problem. Experimental results demonstrate the use of the implemented prototype system to locate the boom, stick and bucket of an...

Abstract—Hitting and batting tasks, such as tennis forehands, ping-pong strokes, or baseball batting, depend on predictions where the ball can be intercepted and how it can properly be returned to the opponent. These predictions get more accurate over time, hence the behaviors need to be continuously modified. As a result, movement templates with a learned global shape need to be adapted during the execution so that the racket reaches a target position and velocity that will return the ball over to the other side of the net or court. It requires altering learned movements to hit a varying target with the necessary velocity at a specific instant in time. Such a task cannot be incorporated straightforwardly in most movement representations suitable for learning. For example, the standard formulation of the dynamical system based motor primitives (introduced by Ijspeert et al. [1]) does not satisfy this property despite their flexibility which has allowed learning tasks ranging from locomotion to kendama. In order to fulfill this requirement, we reformulate the Ijspeert framework to incorporate the possibility of specifying a desired hitting point and a desired hitting velocity while maintaining all advantages of the original formulation. We show that the proposed movement template formulation works well in two scenarios, i.e., for hitting a ball on a string with a table tennis racketataspecifiedvelocityandforreturningballslaunchedby a ball gun successfully over the net using forehand movements. All experiments were carried out on a Barrett WAM using a four camera vision system. I.

Methods for designing and building perceptual agents should be clean, powerful and practical. But no methodology satisfies all three criteria, yet. Our methodologies are evolving dialectically. The symbolic methods of Good Old-Fashioned Artificial Intelligence and Robotics (GOFAIR) constitute the original thesis. The antithesis is reactive Insect AI. The emerging synthesis, Situated Agents, needs formal rigor and practical tools. A robot is a hybrid intelligent dynamical system, consisting of a controller coupled to its body. The Constraint Net (CN) model of Zhang and Mackworth is a unitary framework for building hybrid intelligent systems as situated agents. Most other robot design methodologies use hybrid models of hybrid systems, awkwardly combining offline computational models of high-level perception, reasoning and planning with online models of low-level sensing and control. In CN, the designer specifies the robot's vision, control and motor systems uniformly as online systems. T...

We accept this thesis as conforming to the required standard