DMCA
Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study
| Citations: | 41 - 7 self |
BibTeX
@MISC{Borji_quantitativeanalysis,
author = {Ali Borji and Dicky N. Sihite and Laurent Itti},
title = {Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract—Visual attention is a process that enables biological and machine vision systems to select the most relevant regions from a scene. Relevance is determined by two components: 1) top-down factors driven by task and 2) bottom-up factors that highlight image regions that are different from their surroundings. The latter are often referred to as “visual saliency”. Modeling bottom-up visual saliency has been the subject of numerous research efforts during the past 20 years, with many successful applications in computer vision and robotics. Available models have been tested with different datasets (e.g., synthetic psychological search arrays, natural images or videos) using different evaluation scores (e.g., search slopes, comparison to human eye tracking) and parameter settings. This has made direct comparison of models difficult. Here we perform an exhaustive comparison of 35 state-of-the-art saliency models over 54 challenging synthetic patterns, 3 natural image datasets, and 2 video datasets, using 3 evaluation scores. We find that although model rankings vary, some models consistently perform better. Analysis of datasets reveals that existing datasets are highly center-biased, which influences some of the evaluation scores. Computational complexity analysis shows that some models are very fast, yet yield competitive eye movement prediction accuracy. Different models often have common easy/difficult stimuli. Furthermore, several concerns in visual saliency modeling, eye movement datasets, and evaluation scores are discussed and insights for future work are provided. Our study allows one to assess the state-of-the-art, helps organizing this rapidly growing field, and sets a unified comparison framework for gauging future efforts, similar to the PASCAL VOC challenge in the object recognition and detection domains. Index Terms—Visual attention, Visual saliency, Bottom-up attention, Eye movement prediction, Model comparison.
Keyphrases
visual saliency modeling human-model agreement quantitative analysis evaluation score comparative study visual saliency search slope state-of-the-art saliency model different datasets computational complexity analysis different model object recognition pascal voc challenge exhaustive comparison detection domain bottom-up factor human eye relevant region image region eye movement prediction index term visual attention future work model ranking machine vision system common easy difficult stimulus top-down factor synthetic pattern available model abstract visual attention many successful application future effort video datasets several concern eye movement datasets unified comparison framework different evaluation score bottom-up visual saliency natural image computer vision synthetic psychological search array datasets reveals numerous research effort competitive eye movement prediction accuracy parameter setting bottom-up attention direct comparison model comparison natural image datasets
