Citation Context

...mple, the gestures made by baseball coaches involve complex combinations of body and hand signals. The use of multiple views in human action recognition has been shown to improve recognition accuracy =-=[1, 3]-=-. Evidence from psychological experiments provides theoretical justification [25], showing that people reason about interaction between views (i.e., causal inference) when given combined input signals...

Citation Context

...sus from various features for reordering the documents and boosting the retrieval precision. There are two general approaches along this direction: visual pattern mining [8] and multi-modality fusion =-=[1]-=-, [2]. The former mines the recurrent patterns, either explicitly or implicitly, from initial search results and then moves up the ranks of visually similar documents. Random walk [9], for instance, p...

Citation Context

...ddle-level fusion (saliency curves) or c) high-level fusion (curve features, salient segments, events). The process of combining feature or saliency curves can in general be linear or nonlinear [10], =-=[71]-=-, have memory, or vary with time. In addition, representations from heterogeneous modalities may require preprocessing, normalization and alignment prior toEVANGELOPOULOS et al.: MULTIMODAL SALIENCY ...

Citation Context

...iment on implicit tag validation using facial expressions. They use geometric features obtained from a particle filter tracker with a Hidden Markov Model to assess the correctness of tags displayed alongside images from the participants’ facial expressions. On average, 54% of the trials are correctly classified. 2.4. Multi-modal approaches In general, approaches for modality fusion can be classified into two broad categories, namely, feature fusion (or early integration) and decision fusion (or late integration) [31].Some works have also attempted a combination of both methods in hybrid fusion[32]. In feature-level fusion, the features extracted from signals of different modalities are concatenated to form a composite feature vector and then inputted to a recognizer. In decision fusion, on the other hand, each modality is processed independently by the corresponding classifier and the outputs of the classifiers are combined to yield the final result. Various methods can be used for decision-level fusion, such as simple rule-based methods (for instance taking the sum or product of class probabilities) to classifier-based approaches, where a meta-classifier is trained taking the decision...

Citation Context

...1) i=1,...,r which has the best performance on a given validation set. We now consider the full use of the meta detectors. Strategy II: Learn to Fuse. We adopt linear fusion due to its widespread use =-=[1]-=-, and define the fused detector as svm gnn (40.78, -73.96) gvnn (40.78, -73.96) gisnb (40.78, -73.96) GIS tags arts church library school ... HΛ(x,ω) = r∑ λi ·Hi(x,ω), (2) i=1 where Λ = {λi} are the w...

Citation Context

...other elements. Nevertheless, to make our discussion general, we consider combining annotation functions driven by multiple sources of evidence. To that end, let {g1(x,w),...,gt(x,w)}, with gj(x,w) ∈ =-=[0,1]-=-, j = 1,...,t, be a set of such image annotation functions. As combining these functions can be viewed as a multi-modal fusion problem, we choose to use a linear weighted sum, an effective strategy fo...

Citation Context

...types of fusion are needed in different steps of the experimentation.sDepending on the point the fusion is carried out, it is called early fusion (at featureslevel) or late fusion (at decision level) =-=[8]-=-. Moreover, fusion can be applied amongsresources from different modes (text and image), or the same (different sources ofstext). Late fusion algorithms are used when fusing multiple modalities in the...