research-article

Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations

Authors:

Claudio M. Privitera,

Lawrence W. StarkAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 22, Issue 9

Pages 970 - 982

https://doi.org/10.1109/34.877520

Published: 01 September 2000 Publication History

Abstract

Many machine vision applications, such as compression, pictorial database querying, and image understanding, often need to analyze in detail only a representative subset of the image, which may be arranged into sequences of loci called regions-of-interest, ROIs. We have investigated and developed a methodology that serves to automatically identify such a subset of aROIs (algorithmically detected ROIs) using different Image Processing Algorithms, IPAs, and appropriate clustering procedures. In human perception, an internal representation directs top-down, context-dependent sequences of eye movements to fixate on similar sequences of hROIs (human identified ROIs). In this paper, we introduce our methodology and we compare aROIs with hROIs as a criterion for evaluating and selecting bottom-up, context-free algorithms. An application is finally discussed.

References

[1]

T.A. Bahill and L.W. Stark, “Trajectories of Saccadic Eye Movements,” Scientific Am., vol. 240, pp. 84-93, 1979.

[2]

S. Brandt and L.W. Stark, “Spontaneous Eye Movements during Visual Imagery Reflect the Content of the Visual Scene,” J. Cognitive Neuroscience, vol. 9, no. 1, pp. 27-38, 1997.

Digital Library

[3]

J. Canny, “A Computational Approach to Edge Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 679-698, 1986.

Digital Library

[4]

I. Daubechies, Ten Lectures on Wavelets. Philadelphia: SIAM, 1992.

Digital Library

[5]

C. De Vleeschouwer X. Marichal T. Delmot and B. Macq, “A Fuzzy Logic System Able to Detect Interesting Areas of a Video Sequence,” Proc. SPIE, vol. 3, 016, pp. 234-245, 1997.

[6]

V. Di Gesú and C. Valenti, “The Discrete Symmetry Transform in Computer Vision,” Technical Report 011-95, Laboratory for Computer Science (DMA), Univ. of Palermo, 1995.

[7]

L. Fisher, Fixed Effects Analysis of Variance. New York: Academic Press, 1978.

[8]

L. Itti C. Koch and E. Niebur, “A Model of Saliency-Based Visual Attention for Rapid Scene Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1,254-1,259, Nov. 1998.

Digital Library

[9]

P.J. Locher and C.F. Nodine, “Symmetry Catches the Eye,” Eye Movements: From Physiology to Cognition, pp. 353-361, 1987.

[10]

S.K. Mannan K.H. Ruddock and D.S. Wooding, “The Relationship between the Locations of Spatial Features and Those of Fixations Made during Visual Examination of Briefly Presented Images,” Spatial Vision, vol. 10, no. 3, pp. 165-188, 1996.

[11]

E. Niebur and C. Koch, “Control of Selective Visual Attention: Modeling the 'Where' Pathway,” Advances in Neural Information Processing Systems, D.S. Touretzky, M.C. Mozer, and M.E. Hasselmo, eds., vol. 8, pp. 802-808, MIT Press, 1996.

[12]

D. Noton and L.W. Stark, “Scanpaths in Eye Movements During Pattern Perception,” Science, vol. 17, no. 1, pp. 308-311, 1971.

[13]

W. Osberger and A.J. Maeder, “Automatic Identification of Perceptually Important Regions in an Image,” Proc. Int'l Conf. Pattern Recognition, pp. 17-20, 1998.

Digital Library

[14]

R. Plamondon and C.M. Privitera, “The Segmentation of Cursive Handwriting: An Approach Based on Off-Line Recovery of the Motor-Temporal Information,” IEEE Trans. Image Processing, vol. 8, no. 1, pp. 80-91, 1999.

Digital Library

[15]

C.M. Privitera M. Azzariti and L.W Stark, “Locating Regions-of-Interest for the Mars Rover,” Int'l J. Remote Sensing, 2000. to appear.

[16]

C.M. Privitera and L.W. Stark, “Evaluating Image Processing Algorithms That Predict Regions of Interest,” Pattern Recognition Letters, vol. 19, no. 1, pp. 1,037-1,043, 1998.

Digital Library

[17]

C.M. Privitera and L.W. Stark, “Focused JPEG Encoding Based upon Automatic Preidentified Regions of Interest,” Proc. SPIE, vol. 3, 644, pp. 552-558, 1999.

[18]

D. Reisfeld H. Wolfson and Y. Yeshurun, “Context-Free Attentional Operators: The Generalized Symmetry Transform,” Int'l J. Computer Vision, vol. 14, pp. 119-130, 1995.

Digital Library

[19]

W. Richards and L. Kaufman, “Centre-of-Gravity Tendencies for Fixations and Flow Patterns,” Perception and Psychology, vol. 5, pp. 81-84, 1969.

[20]

O. Rioul and P. Duhamel, “Fast Algorithms for Discrete and Continuous Wavelet Transforms,” IEEE Trans. Information Theory, vol. 38, no. 2, pp. 569-586, 1992.

[21]

L.W. Stark and Y. Choi, “Experimental Metaphysics: The Scanpath as an Epistemological Mechanism,” Visual Attention and Cognition, W.H. Zangemeister, H.S. Stiehl, and C. Freksa, eds., pp. 3-69, 1996.

[22]

L.W. Stark and C.M. Privitera, “Top-Down and Bottom-Up Image Processing,” Proc. of IEEE Int'l Conf. Neural Networks, vol. 4, pp. 2,294-2,299, June 1997.

[23]

L.W. Stark C.M. Privitera H. Yang M. Azzariti Y.F. Ho A. Chan C. Krischer and A. Weinberger, “Scanpath Memory Binding: Multiple Read-Out Experiments,” Proc. SPIE, vol. 3, 644, pp. 495-510, 1999.

[24]

P.P. Vaidyanathan, Multirate System and Filter Banks. Englewood Cliffs, N.J.: Prentice Hall, 1993.

Digital Library

[25]

A.L. Yarbus, Eye Movements and Vision. New York: Plenum Press, 1967.

[26]

W.H. Zangemeister K. Sherman and L.W. Stark, “Evidence for Global Scanpath Strategy in Viewing Abstract Compared with Realistic Images,” Neuropsychologia, vol. 33, no. 8, pp. 1,009-1,025, 1995.

[27]

J. Zhao Y. Shimazu K. Ohta R. Hayasaka and Y. Matsushita, “A JPEG Codec Adaptive to the Relative Importance of Regions in an Image,” IEEE Trans. Information Processing Soc. Japan, vol. 38, no. 8, pp. 1,531-1,542, 1997.

Cited By

Lou JLin HYoung PWhite RYang ZShelmerdine SMarshall DSpezi EPalombo MLiu H(2024)Predicting Radiologists' Gaze With Computational Saliency Models in Mammogram ReadingIEEE Transactions on Multimedia10.1109/TMM.2023.326355326(256-269)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3263553
Byrne SMaquiling VReynolds APolonio LCastner NKasneci E(2023)Exploring the Effects of Scanpath Feature Engineering for Supervised Image Classification ModelsProceedings of the ACM on Human-Computer Interaction10.1145/35911307:ETRA(1-18)Online publication date: 18-May-2023
https://dl.acm.org/doi/10.1145/3591130
Fuhl WZabel SHarbig TMoldt JWietek THerrmann-Werner ANieselt K(2023)Area of interest adaption using feature importanceProceedings of the 2023 Symposium on Eye Tracking Research and Applications10.1145/3588015.3588422(1-7)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588015.3588422
Show More Cited By

Index Terms

Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

Image-type dependent eigen-regions-of-interest define conspicuity operators for predicting human scanpath fixation

Top-down informativeness elaboration and bottom-up conspicuity processing are intimately interconnected in visual perception. An internal cognitive model of the external world must necessarily control not only our recognition but also the scanpath ...
Human Visual Scanpath Prediction Based on RGB-D Saliency
ICIGP '18: Proceedings of the 2018 International Conference on Image and Graphics Processing

Human visual perception is considered as a dynamic process of information acquisition, while the visual scanpath can clearly reflect the shift of our eye fixations. In the previous study of visual attention, researchers generally do the saliency ...
Video game design using an eye-movement-dependent model of visual attention

Eye movements can be used to infer the allocation of covert attention. In this article, we propose to model the allocation of attention in a task-dependent manner based on different eye movement conditions, specifically fixation and pursuit. We show ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 22, Issue 9

September 2000

142 pages

ISSN:0162-8828

Editor:
Kevin Bowyer
Univ. of South Florida, Tampa

Issue’s Table of Contents

Copyright © Copyright © 2000 IEEE. All Rights Reserved.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 September 2000

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

112
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lou JLin HYoung PWhite RYang ZShelmerdine SMarshall DSpezi EPalombo MLiu H(2024)Predicting Radiologists' Gaze With Computational Saliency Models in Mammogram ReadingIEEE Transactions on Multimedia10.1109/TMM.2023.326355326(256-269)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3263553
Byrne SMaquiling VReynolds APolonio LCastner NKasneci E(2023)Exploring the Effects of Scanpath Feature Engineering for Supervised Image Classification ModelsProceedings of the ACM on Human-Computer Interaction10.1145/35911307:ETRA(1-18)Online publication date: 18-May-2023
https://dl.acm.org/doi/10.1145/3591130
Fuhl WZabel SHarbig TMoldt JWietek THerrmann-Werner ANieselt K(2023)Area of interest adaption using feature importanceProceedings of the 2023 Symposium on Eye Tracking Research and Applications10.1145/3588015.3588422(1-7)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588015.3588422
Sandnes F(2023)Towards “Image Reflow” on the Web: Avoiding Excessive Panning of Magnified Images by Multiplexing Automatically Cropped Regions of InterestHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42293-5_29(315-319)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-42293-5_29
Beelders TAl Madi NSharif B(2022)Eye-tracking analysis of source code reading on a line-by-line basisProceedings of the Tenth International Workshop on Eye Movements in Programming10.1145/3524488.3527364(1-7)Online publication date: 19-May-2022
https://dl.acm.org/doi/10.1145/3524488.3527364
Kurzhals K(2021)Image-Based Projection Labeling for Mobile Eye TrackingACM Symposium on Eye Tracking Research and Applications10.1145/3448017.3457382(1-12)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448017.3457382
Xia CHan JZhang D(2021)Evaluation of Saccadic Scanpath Prediction: Subjective Assessment Database and Recurrent Neural Network Based MetricIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2020.300216843:12(4378-4395)Online publication date: 2-Nov-2021
https://dl.acm.org/doi/10.1109/TPAMI.2020.3002168
Khan ALi JHaq ANazir SAhmad NVarish NMalik APatel SAli R(2020)Partial Observer Decision Process Model for Crane-Robot ActionScientific Programming10.1155/2020/63493422020Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1155/2020/6349342
Greenberg SBensimon MPodeneshko YGens A(2020)Fusion of visual salience maps for object acquisitionIET Computer Vision10.1049/iet-cvi.2019.062414:4(113-121)Online publication date: 8-Apr-2020
https://dl.acm.org/doi/10.1049/iet-cvi.2019.0624
Sharafi ZSharif BGuéhéneuc YBegel ABednarik RCrosby M(2020)A practical guide on conducting eye tracking studies in software engineeringEmpirical Software Engineering10.1007/s10664-020-09829-425:5(3128-3174)Online publication date: 1-Sep-2020
https://dl.acm.org/doi/10.1007/s10664-020-09829-4
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents