## Spike Train Driven Dynamical Models for Human Actions

Citations: | 1 - 0 self |

### BibTeX

@MISC{Raptis_spiketrain,

author = {Michalis Raptis and Kamil Wnuk and Stefano Soatto},

title = {Spike Train Driven Dynamical Models for Human Actions},

year = {}

}

### OpenURL

### Abstract

We investigate dynamical models of human motion that can support both synthesis and analysis tasks. Unlike coarser discriminative models that work well when action classes are nicely separated, we seek models that have finescale representational power and can therefore model subtle differences in the way an action is performed. To this end, we model an observed action as an (unknown) linear time-invariant dynamical model of relatively small order, driven by a sparse bounded input signal. Our motivating intuition is that the time-invariant dynamics will capture the unchanging physical characteristics of an actor, while the inputs used to excite the system will correspond to a causal signature of the action being performed. We show that our model has sufficient representational power to closely approximate large classes of non-stationary actions with significantly reduced complexity. We also show that temporal statistics of the inferred input sequences can be compared in order to recognize actions and detect transitions between them. 1.

### Citations

3666 |
Convex Optimization
- Boyd, Vandenberghe
- 2004
(Show Context)
Citation Context ... unweighted ℓ1 regularized problem, where w = [w0, . . . , wN−1] T . Afterward, we introduce a new variable z ∈ R N , a new equality constraint z = HD −1 ũ − ˜y, and make the box constraints implicit =-=[5]-=-. minimize −w≼ ũ ≼w,z The dual function of (7) is: g(ν) = where q + i ∑ |ũi| zT N−1 z + λ i=0 subject to: z = HD −1 ũ − ˜y. (7) inf −w≼ũ≼w,z (zT z + λ‖ũ‖1 + ν T (HD −1 ũ − ˜y − z)) = νT ν 4 − w T ((D ... |

1832 | Regression shrinkage and selection via the lasso
- Tibshirani
- 1994
(Show Context)
Citation Context ..., i = 0, . . . , N − 1 N−2 ∑ i=0 ‖CA i B‖1 ≤ µ where ˆ Y is the observed time series. It is well known that the minimization in (3) is NP-hard, thus we relax the problem to a weighted ℓ1 minimization =-=[25]-=-: minimize U,X0,A,B,C ‖ ˆ Y − ΓXo − HU‖ 2 N−1 2 + λ ∑ wi|ui| i=0 subject to: |ui| ≤ 1 , i = 0, . . . , N − 1 N−2 ∑ i=0 ‖CA i B‖1 ≤ µ. The form above adds a regularizer term, with λ serving as the trad... |

927 |
Emergence of Simple-Cell Receptive Field Properties by Learning a Sparse Code for Natural Images
- Olshausen, Field
- 1996
(Show Context)
Citation Context ...as the tradeoff between accuracy of fit and sparsity. 3.1. Alternating Minimization Our approach to solve (4) is similar in spirit to algorithms for learning dictionaries to sparsely represent images =-=[16]-=-. In our case, however, the dictionary is the impulse response of the linear dynamical system H. (3) (4) Algorithm: 1. Select the order of the system 1 : n 2. Initialize a random sparse input U satisf... |

355 | Recognizing action at a distance
- Efros, Berg, et al.
- 2003
(Show Context)
Citation Context ...s trivial, we focus on the second problem of learning dynamical models from the time series. For some tasks, such as classification of distinctive motions, purely discriminative models are sufficient =-=[9]-=-. Some benchmark datasets can even be classified reliably taking into account as little information as local shape and optical flow in a single frame [23]. However, in situations where the temporal or... |

288 | Dynamic textures
- Doretto, Chiuso, et al.
- 2003
(Show Context)
Citation Context ...of Actions Once a model is learned, the inputs and parameters of each dynamical system are directly available and can be controlled purposefully in ways similar to Doretto et al. for dynamic textures =-=[8]-=-. For example, we can change the intensity of the motion by scaling the C matrix of the system. This type of creative editing of the dynamics and input can result in interesting variations on an origi... |

279 | Animating human athletics
- HODGINS, WOOTEN, et al.
- 1995
(Show Context)
Citation Context ... not provide information which can directly be used for classification or segmentation of the modeled motion. Physically based nonlinear temporal models have also been used to synthesize human motion =-=[10, 11]-=-. However, the process of concatenating “basic” controllers becomes too complex for most actions of interest. In our case we assume a single linear time-invariant model. We show that by changing the a... |

193 | Motion warping
- WITKIN, POPOVIC
- 1995
(Show Context)
Citation Context ...s is a valid method of evaluating what we capture, it is not the key goal of our model. Thus we do not focus on adding any kinematic or smoothness constraints, as is often done in graphics literature =-=[30, 1]-=- to generate lifelike motions. Finally, we compute that on average, in FutureLight, 78.84% of the input signal values are zero, confirming that the inferred signal is sparse. An advantage that comes w... |

153 | An Interior-Point Method for Large-Scale l1 Regularized Logistic Regression
- Koh, Kim, et al.
- 2007
(Show Context)
Citation Context ...m. The multiplication of a Toeplitz matrix with a vector can be performed in O(N log N) instead of O(N 2 ). In our experiments we use the truncated Newton interior-point method proposed by Kim et al. =-=[13]-=-, modified according to the specific constraints of our formulation. In the situation where the output is multivariate, H can be represented with p Toeplitz matrices to maintain efficiency during mult... |

145 | Motion synthesis from annotations
- Arikan, Forsyth, et al.
- 2003
(Show Context)
Citation Context ...s is a valid method of evaluating what we capture, it is not the key goal of our model. Thus we do not focus on adding any kinematic or smoothness constraints, as is often done in graphics literature =-=[30, 1]-=- to generate lifelike motions. Finally, we compute that on average, in FutureLight, 78.84% of the input signal values are zero, confirming that the inferred signal is sparse. An advantage that comes w... |

145 | Parametric hidden Markov models for gesture recognition
- Wilson, Bobick
- 1999
(Show Context)
Citation Context ...ne scale prediction, models with generative capability and greater representational accuracy are useful. Since the discrete multinomial state of generative models, such as Hidden Markov Models (HMMs) =-=[31, 29, 12]-=-, experience an exponential increase in parameters as more signal history is encoded, we favor dynamic models with continuous latent variables to pursue the desired level of detail in action represent... |

130 | Composable Controllers for Physics-based Character Animation
- Faloutsos, Panne, et al.
- 2001
(Show Context)
Citation Context ... not provide information which can directly be used for classification or segmentation of the modeled motion. Physically based nonlinear temporal models have also been used to synthesize human motion =-=[10, 11]-=-. However, the process of concatenating “basic” controllers becomes too complex for most actions of interest. In our case we assume a single linear time-invariant model. We show that by changing the a... |

111 | Learning switching linear models of human motion
- Pavlovic, Rehg, et al.
- 2000
(Show Context)
Citation Context ... simpler regular actions. This motivates the use of switched-linear dynamical systems (SLDS), in which changes of the model parameters enhance the ability of the model to capture more complex motions =-=[19, 15]-=-. In [18], an SLDS approach was proposed where only the zeros of the transfer function were allowed to change across actions and an HMM was used to drive these changes. Works with a similar spirit hav... |

111 |
Subspace Identification for Linear Systems: Theory, Implementation, Applications
- Overschee, Moor
- 1996
(Show Context)
Citation Context ... ΓX0 − HU‖ 2 N−1 2 + λ i=0 ∑ wi|ui| subject to: |ui| ≤ 1 i = 0, . . . , N − 1 (5) For estimating the A and C matrices of the LDS we use the subspace identification algorithm for deterministic systems =-=[26]-=- with the constraint that A must be stable. For this purpose we adopt the method [24], which incrementally adds constraints to a quadratic program to improve the stability of the estimated system matr... |

93 |
Enhancing sparsity by reweighted l1 minimization
- Candes, Wakin, et al.
- 2008
(Show Context)
Citation Context ...cing Sparsity The sparsity of the result obtained by solving a uniform weighted ℓ1 - regularized least-squares formulation (5) can be further enhanced by incorporating an iterative reweighting scheme =-=[6]-=-. Step 3(b) of the algorithm above is thus modified as follows: 1. Initialize the weights : w (0) i = 1, i = 0, . . . , N − 1. 2. Solve the weighted ℓ1 minimization problem U (l) = argmin‖ ˆ Y − ΓXo −... |

87 | Gaussian process dynamical models for human motion
- Wang, Fleet, et al.
- 2008
(Show Context)
Citation Context ...d as a convex optimization problem by [17], and as identification of homogeneous polynomials by [27]. Yet another perspective on capturing the non-stationarity of human actions are Gaussian processes =-=[28]-=-. These models learn a nonlinear mapping from the observation space into a latent space and a nonlinear system in the latent space. A downside of this approach is that it does not provide information ... |

86 | Segmenting motion capture data into distinct behaviors
- Barbic, Safonova, et al.
(Show Context)
Citation Context ...Fig. 2 also provides a comparison of our input-based segmentation results with existing algorithms for segmenting temporal data that operate directly on observed values. We compare with Barbič et al. =-=[2]-=-, who proposed a change detection algorithm based on the reprojection error on the principal components computed in a sequential fashion. Their algorithm detects changes in motions relatively accurate... |

84 | Gool, “Action snippets: how many frames does human action recognition require
- Schindler, Van
- 2008
(Show Context)
Citation Context ...s, purely discriminative models are sufficient [9]. Some benchmark datasets can even be classified reliably taking into account as little information as local shape and optical flow in a single frame =-=[23]-=-. However, in situations where the temporal order of motions is significant (or perhaps the only discriminative information), or to address more subtle queries such as long-term or fine scale predicti... |

66 | Segmentation of multivariate mixed data via lossy coding and compression
- Ma, Derksen, et al.
- 2007
(Show Context)
Citation Context ...m -1 to 1. To encode information from the inputs to all of the body’s systems the histograms for each limb and torso are stacked to create a frame descriptor. We then use the lossy coding approach of =-=[14]-=- to produce an unsupervised segmentation. To encourage temporal coherence, we initially restrict merging to only neighboring segments, using a low ρ distortion parameter (ρ = 25). This also significan... |

43 | Impact of dynamic model learning on classification of human motion
- Pavlovic, Rehg
- 2000
(Show Context)
Citation Context ...tention in the machine learning and vision communities. Our model falls into the class of linear dynamical systems, where the task of motion modeling has been posed as a system identification problem =-=[4, 20]-=-. Up until now the LDS literature in human motion has assumed a stochastic input with a known distribution, which limits the representational capability to simpler regular actions. This motivates the ... |

22 | Time series classification using mixed-state dynamic Bayesian networks
- Pavlovic, Frey, et al.
- 1999
(Show Context)
Citation Context ...lar actions. This motivates the use of switched-linear dynamical systems (SLDS), in which changes of the model parameters enhance the ability of the model to capture more complex motions [19, 15]. In =-=[18]-=-, an SLDS approach was proposed where only the zeros of the transfer function were allowed to change across actions and an HMM was used to drive these changes. Works with a similar spirit have used sw... |

21 | A Constraint Generation Approach to Learning Stable Linear Dynamical Systems
- Siddiqi, Boots, et al.
- 2007
(Show Context)
Citation Context ... estimating the A and C matrices of the LDS we use the subspace identification algorithm for deterministic systems [26] with the constraint that A must be stable. For this purpose we adopt the method =-=[24]-=-, which incrementally adds constraints to a quadratic program to improve the stability of the estimated system matrix. Having estimated A and C, the estimation of B and X0 is the least-squares solutio... |

19 | Classification and Recognition of Dynamical Models: The Role of Phase
- Bissacco, Chiuso, et al.
- 2007
(Show Context)
Citation Context ...ssian). These approaches were successful at capturing observations with second-order stationary statistics, and therefore worked well for modeling quasi-repetitive actions such as walking and running =-=[3]-=-. However, the limitations of these models become quickly apparent when one considers more complex non-stationary sequences, e.g. Fig. 1. Our goal in this work is to be able to capture such non-statio... |

19 |
Learning and inference in parametric switching linear dynamical systems
- Oh, Rehg, et al.
- 2005
(Show Context)
Citation Context ... simpler regular actions. This motivates the use of switched-linear dynamical systems (SLDS), in which changes of the model parameters enhance the ability of the model to capture more complex motions =-=[19, 15]-=-. In [18], an SLDS approach was proposed where only the zeros of the transfer function were allowed to change across actions and an HMM was used to drive these changes. Works with a similar spirit hav... |

18 |
Human action recognition by sequence of movelet codewords
- Feng, Perona
- 1986
(Show Context)
Citation Context ...ne scale prediction, models with generative capability and greater representational accuracy are useful. Since the discrete multinomial state of generative models, such as Hidden Markov Models (HMMs) =-=[31, 29, 12]-=-, experience an exponential increase in parameters as more signal history is encoded, we favor dynamic models with continuous latent variables to pursue the desired level of detail in action represent... |

11 | Searching for complex human activities with no visual examples
- ˙Ikizler, Forsyth
(Show Context)
Citation Context ...ne scale prediction, models with generative capability and greater representational accuracy are useful. Since the discrete multinomial state of generative models, such as Hidden Markov Models (HMMs) =-=[31, 29, 12]-=-, experience an exponential increase in parameters as more signal history is encoded, we favor dynamic models with continuous latent variables to pursue the desired level of detail in action represent... |

7 |
Recursive identification of switched ARX systems
- Vidal
(Show Context)
Citation Context ...ed by detecting changes of the coefficients of the AR model. The identification of SAR has been addressed as a convex optimization problem by [17], and as identification of homogeneous polynomials by =-=[27]-=-. Yet another perspective on capturing the non-stationarity of human actions are Gaussian processes [28]. These models learn a nonlinear mapping from the observation space into a latent space and a no... |

4 | 2001a, Some Algorithmic aspects of Subspace Identification with
- Chiuso, Picci
(Show Context)
Citation Context ... constraints to a quadratic program to improve the stability of the estimated system matrix. Having estimated A and C, the estimation of B and X0 is the least-squares solution of the simulation error =-=[7]-=-. 3.2. Enhancing Sparsity The sparsity of the result obtained by solving a uniform weighted ℓ1 - regularized least-squares formulation (5) can be further enhanced by incorporating an iterative reweigh... |

4 | Sequential sparsification for change detection
- Ozay, Sznaier, et al.
- 2008
(Show Context)
Citation Context ...SAR) systems to model videos. Video segmentation is achieved by detecting changes of the coefficients of the AR model. The identification of SAR has been addressed as a convex optimization problem by =-=[17]-=-, and as identification of homogeneous polynomials by [27]. Yet another perspective on capturing the non-stationarity of human actions are Gaussian processes [28]. These models learn a nonlinear mappi... |

4 |
Flexible Dictionaries for Action Recognition
- Raptis, Wnuk, et al.
- 2008
(Show Context)
Citation Context ...into a sequence of labels. To take into account the temporal alignment of labels but also utilize support vector machines (SVM) with RBF kernel, we use the Smith-Waterman based technique described in =-=[21]-=- for classification. Classification performance is evaluated with a leave-one out cross validaDance Jump Sit Run Walk Dance 24 2 2 3 Jump 2 11 1 Sit 1 34 Run 3 3 23 1 Walk 5 43 Table 3. Confusion Matr... |

2 | Classifying human dynamics without contact forces
- Bissacco, Soatto
(Show Context)
Citation Context ...tention in the machine learning and vision communities. Our model falls into the class of linear dynamical systems, where the task of motion modeling has been posed as a system identification problem =-=[4, 20]-=-. Up until now the LDS literature in human motion has assumed a stochastic input with a known distribution, which limits the representational capability to simpler regular actions. This motivates the ... |

2 |
Chaotic invariants for human action recognition
- Saad, Arslan, et al.
- 2007
(Show Context)
Citation Context ...the search direction is computed as an approximate solution to the Newton system, using Preconditioned Conjugate Gradient [13]. 5. Experimental Evaluation 5.1. Datasets The FutureLight action dataset =-=[22]-=- is a collection of 5 actions, performed with significant intra and inter-class variations: “Dance”, “Jump”, “Sit”, “Run”, and “Walk”. The durations of captured actions vary from 100 to over 800 frame... |