## The Computational Perception of Scene Dynamics (1995)

### Cached

### Download Links

- [www.cs.toronto.edu]
- [ftp.cs.toronto.edu]
- [ftp.cs.toronto.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | Computer Vision and Image Understanding |

Citations: | 46 - 3 self |

### BibTeX

@INPROCEEDINGS{Mann95thecomputational,

author = {Richard Mann and Allan Jepson and Jeffrey Mark Siskind},

title = {The Computational Perception of Scene Dynamics},

booktitle = {Computer Vision and Image Understanding},

year = {1995},

pages = {528--539}

}

### Years of Citing Articles

### OpenURL

### Abstract

Understanding observations of interacting objects requires one to reason about the force-dynamic relations between objects. We present an implemented computational theory that derives force-dynamic interpretations directly from camera input. Interpretations are expressed in terms of assertions about the kinematic and dynamic properties of objects. The feasibility of interpretations can be determined relative to Newtonian mechanics by a reduction to linear programming. Multiple feasible solutions are compared using a preference hierarchy to select plausible interpretations. We provide computational examples to demonstrate that our ontology is sufficiently rich to describe a wide variety of image sequences. KEYWORDS: Motion understanding, Scene dynamics, Perceptual inference, Knowledgebased perception, Domain theory, View-based representations. Submitted. 1 Introduction Both AI and psychology researchers have argued for the need to represent "causal" information about the world in ...

### Citations

1041 |
Introduction to Linear and Nonlinear Programming
- Luenberger
- 1973
(Show Context)
Citation Context ... since the observed motion will never be exactly zero. Since all of the above equations and inequalities are linear, dynamic feasibility can be reduced to a feasibility test using linear programming (=-=Luenberger, 1984). 6-=- Preferences As described in x3, we have a fixed set of elementary preference relations, namely ffl P bodymotor (o) : :BodyMotor(o) �� BodyMotor(o); ffl P linearmotor (c) : :LinearMotor(o 1 ; o 2 ... |

655 |
H.Classical Mechanics
- Goldstein
- 1980
(Show Context)
Citation Context ...th linear programming. This test is valid for both two and three dimensional scene models. For rigid bodies under continuous motion, the dynamics are described by the Newton-Euler equations of motion =-=[9]-=- which relate the total applied force and torque to the observed accelerations of the objects. Given a scene with convex polygonal object parts, we can represent the forces between contacting parts by... |

322 | Pointwise circumscription, in
- Lifschitz
- 1988
(Show Context)
Citation Context ...ions, then we check the assertions at the next lower priority, and so on. This approach, based upon prioritised ordering of elementary preference relations, is similar to prioritised circumscription (=-=Lifschitz, 1985-=-). To find maximally-preferred models, we search the space of possible interpretations. We perform a breadth-first search, starting with the empty set of assertions, incrementally adding new assertion... |

216 |
Robot Dynamics Algorithms
- Featherstone
- 1987
(Show Context)
Citation Context ...possible interpretations we must first construct a set of admissible assertions. Given that the allowable forces between objects depend on the contact geometry and the relative motion of the objects (=-=Featherstone, 1987-=-), an analysis of the scene kinematics is necessary. Since we do not have exact shape or motion information, however, we need a way to determine which contact relations are possible. In general, the d... |

80 |
Problem-solving with diagrammatic representations
- Funt
- 1980
(Show Context)
Citation Context ...ented `causal' structure using rule-based systems (Fahlman, 1974; Tsotsos, Mylopoulos, Covvey, & Zucker, 1980; Brand, Birnbaum, & Cooper, 1993; Kuniyoshi & Inoue, 1993) and naive physical simulation (=-=Funt, 1980-=-; Joskowicz & Sacks, 1991; Siskind, 1992, 1995). In contrast, our system uses an explicit representation based on Newtonian physics. A number of other systems have used physically-based representation... |

74 | A logical framework for depiction and image interpretation, Artif. lntell. 41 - Reiter, Mackworth - 1989 |

66 | Toward an Assembly Plan from Observation Part I: Task Recognition with Polyhedral Objects - Ikeuchi, Suehiro - 1994 |

59 | Grounding Language in Perception
- Siskind
- 1994
(Show Context)
Citation Context ...patiotemporal features of the input [1, 23,18,4,14].Anumber of other systems have attempted to represent structure in static and dynamic scenes using qualitative physical models or rule-based systems =-=[6, 8, 13, 20, 22, 5]-=-. In contrast to both ? Also at Canadian Institute for Advanced Research. ?? Current address: Department of Electrical Engineering, Technion, Haifa 32000, ISRAELof these approaches, our system uses a... |

58 |
Temporal Scene Analysis: Conceptual descriptions of object movements
- Badler
- 1975
(Show Context)
Citation Context ...tudied extensively for both static and motion domains. Many prior systems have attempted to extract event or conceptual descriptions from image sequences based on spatiotemporal features of the input =-=[1, 23, 18, 4, 14]-=-. A number of other systems have attempted to represent structure in static and dynamic scenes using qualitative physical models or rule-based systems [6, 8, 13, 20, 22, 5]. In contrast to both ? Also... |

47 | Naive physics, event perception, lexical semantics, and language acquisition. Doctoral dissertation, Electrical Engineering and Computer Science Department
- Siskind
- 1992
(Show Context)
Citation Context ...ased systems (Fahlman, 1974; Tsotsos, Mylopoulos, Covvey, & Zucker, 1980; Brand, Birnbaum, & Cooper, 1993; Kuniyoshi & Inoue, 1993) and naive physical simulation (Funt, 1980; Joskowicz & Sacks, 1991; =-=Siskind, 1992-=-, 1995). In contrast, our system uses an explicit representation based on Newtonian physics. A number of other systems have used physically-based representations for scenes in terms of forces in stati... |

43 | Qualitative recognition of ongoing human action sequences
- Kuniyoshi, Inoue
- 1993
(Show Context)
Citation Context ...tudied extensively for both static and motion domains. Many prior systems have attempted to extract event or conceptual descriptions from image sequences based on spatiotemporal features of the input =-=[1, 23,18,4,14]-=-.Anumber of other systems have attempted to represent structure in static and dynamic scenes using qualitative physical models or rule-based systems [6, 8, 13, 20, 22, 5]. In contrast to both ? Also a... |

41 |
S.W.: A Framework for Visual Motion Understanding
- Tsotsos, Mylopoulos, et al.
- 1980
(Show Context)
Citation Context ...tudied extensively for both static and motion domains. Many prior systems have attempted to extract event or conceptual descriptions from image sequences based on spatiotemporal features of the input =-=[1, 23, 18, 4, 14]-=-. A number of other systems have attempted to represent structure in static and dynamic scenes using qualitative physical models or rule-based systems [6, 8, 13, 20, 22, 5]. In contrast to both ? Also... |

33 |
A planning system for robot construction tasks
- Fahlman
- 1974
(Show Context)
Citation Context ...velocity and acceleration, and is not designed to handle impulses and abrupt changes in contact. 8 Related Work A number of prior systems have represented `causal' structure using rule-based systems (=-=Fahlman, 1974-=-; Tsotsos, Mylopoulos, Covvey, & Zucker, 1980; Brand, Birnbaum, & Cooper, 1993; Kuniyoshi & Inoue, 1993) and naive physical simulation (Funt, 1980; Joskowicz & Sacks, 1991; Siskind, 1992, 1995). In co... |

30 |
Classical mechanics. Second edition
- Goldstein
- 1980
(Show Context)
Citation Context ...al results we describe later were produced using a two-dimensional variant of this theory. For rigid bodies under continuous motion, the dynamics are described by the NewtonEuler equations of motion (=-=Goldstein, 1980-=-). For rigid bodies of non-varying mass, the appropriate equations are: F =sp = Msv (2) N = L = Is! + ! \Theta I! (3) The first equation relates the total applied force F to the rate of change of line... |

27 |
Interactive simulation of solid rigid bodies
- Baraff
- 1995
(Show Context)
Citation Context ...eleration. were not used. Finally, while the problem is quite different, our representation of rigid objects borrows heavily from the physical simulation and graphics communities (Featherstone, 1987; =-=Baraff, 1995-=-). 9 Conclusion In this paper we have presented an implemented computational theory that can derive forcedynamic representations directly from camera input. Our system embodies a rich ontology that in... |

25 |
Event models for recognition and natural language descriptions of events in real-world image sequences
- Neumann, Novak
- 1983
(Show Context)
Citation Context ...tudied extensively for both static and motion domains. Many prior systems have attempted to extract event or conceptual descriptions from image sequences based on spatiotemporal features of the input =-=[1, 23, 18, 4, 14]-=-. A number of other systems have attempted to represent structure in static and dynamic scenes using qualitative physical models or rule-based systems [6, 8, 13, 20, 22, 5]. In contrast to both ? Also... |

17 | Priors, preferences, and categorical percepts
- Richards, Jepson, et al.
- 1996
(Show Context)
Citation Context ...tations, we seek interpretations that require, in some specified sense, the weakest properties of the various objects. We use model preference relations, as discussed by Richards, Jepson, and Feldman =-=[19]-=-, to express a suitable ordering on the various interpretations. The basic idea is to compare pairs of interpretations using a prioritised set of elementary preference relations. Our current ontology ... |

15 | A stability test for configurations of blocks - Blum, Griffith, et al. - 1970 |

14 | Sensible scenes: Visual understanding of complex structures through causal analysis
- Brand, Birnbaum, et al.
- 1993
(Show Context)
Citation Context ...patiotemporal features of the input [1, 23,18,4,14].Anumber of other systems have attempted to represent structure in static and dynamic scenes using qualitative physical models or rule-based systems =-=[6, 8, 13, 20, 22, 5]-=-. In contrast to both ? Also at Canadian Institute for Advanced Research. ?? Current address: Department of Electrical Engineering, Technion, Haifa 32000, ISRAELof these approaches, our system uses a... |

13 |
Event calculus
- Borchardt
(Show Context)
Citation Context |

9 | Detecting floor anomalies
- Jenkin, Jepson
(Show Context)
Citation Context ...as a set of two-dimensional convex polygons. To obtain estimates for the object motions we use a view-based tracking algorithm similar to the optical flow and stereo disparity algorithms described in =-=[12, 11]-=-. The input to the tracker consists of the image sequence, a set of object template images (including a polygonal outline for each object), and an estimate for the object positions in the first frame ... |

8 | What's a Percept - Jepson, Richards - 1991 |

6 | Mixture models for optical flow - Jepson, Black - 1993 |

6 |
Detecting oor anomalies
- Jenkin, Jepson
- 1994
(Show Context)
Citation Context ...e as a set of two-dimensional convex polygons. To obtain estimates for the object motions we use a view-based tracking algorithm similar to the optical ow and stereo disparity algorithms described in =-=[12, 11]-=-. The input to the tracker consists of the image sequence, a set of object template images (including a polygonal outline for each object), and an estimate for the object positions in the rst frame of... |

3 | Qualitative motion from visual dynamics - Shavit, Jepson - 1993 |

3 |
Axiomatic Support for Event Perception
- Siskind
- 1994
(Show Context)
Citation Context ...ystems have used physically-based representations for scenes in terms of forces in static scenes (Blum et al., 1970), and changing kinematic relations in time-varying scenes (Ikeuchi & Suehiro, 1994; =-=Siskind, 1994-=-). Our system extends these approaches to consider both kinematic and dynamic relations in time-varying scenes containing rigid objects. Shavit and Jepson (1993) present an approach to classifying mot... |

3 |
Interactive simulation of solid rigid bodies
- Bara
- 1995
(Show Context)
Citation Context ...he objects. Given a scene with convex polygonal object parts, we can represent the forces between contacting parts by a set of forces acting on the vertices of the convex hull of their contact region =-=[7, 2]-=-. Under this simpli cation, the equations of motion for each object can be written as a set of equality constraints which relate the forces and torques at each contact point to the object masses and a... |

3 |
Classical Mechanics. Addison-Wesley, second edition, 1980. Katsushi Ikeuchi and T. Suehiro. Towards an assembly plan from observation, part i: Task recognition with polyhedral objects
- Goldstein
- 1994
(Show Context)
Citation Context ...th linear programming. This test is valid for both two and three dimensional scene models. For rigid bodies under continuous motion, the dynamics are described by the Newton-Euler equations of motion =-=[9]-=- which relate the total applied force and torque to the observed accelerations of the objects. Given a scene with convex polygonal object parts, we can represent the forces between contacting parts by... |

2 |
Mixture models for optical ow
- Jepson, Black
- 1993
(Show Context)
Citation Context ...e as a set of two-dimensional convex polygons. To obtain estimates for the object motions we use a view-based tracking algorithm similar to the optical ow and stereo disparity algorithms described in =-=[12, 11]-=-. The input to the tracker consists of the image sequence, a set of object template images (including a polygonal outline for each object), and an estimate for the object positions in the rst frame of... |

1 | Detecting Floor Anomalies - Jepson - 1994 |

1 |
A stability test for con gurations of blocks
- Blum, th, et al.
- 1970
(Show Context)
Citation Context ...ions. In particular, Ikeuchi and Suehiro [10] and Siskind [21] propose representions of events based on changing kinematic relations in time-varying scenes. Also, closer to our approach, Blum et. al. =-=[3]-=- propose a representation of forces in static scenes. Our system extends these approaches to consider both kinematic and dynamic properties in time-varying scenes containing rigid objects. coke 30 37 ... |