research-article

iScope: personalized multi-modality image search for mobile devices

Authors:

Robert P. DickAuthors Info & Claims

MobiSys '09: Proceedings of the 7th international conference on Mobile systems, applications, and services

Pages 277 - 290

https://doi.org/10.1145/1555816.1555845

Published: 22 June 2009 Publication History

Abstract

Mobile devices are becoming a primary medium for personal information gathering, management, and sharing. Managing personal image data on mobile platforms is a difficult problem due to large data set size, content diversity, heterogeneous individual usage patterns, and resource constraints. This article presents a user-centric system, called iScope, for personal image management and sharing on mobile devices. iScope uses multi-modality clustering of both content and context information for efficient image management and search, and online learning techniques for predicting images of interest. It also supports distributed content-based search among networked devices while maintaining the same intuitive interface, enabling efficient information sharing among people. We have implemented iScope and conducted in-field experiments using networked Nokia N810 portable Internet tablets. Energy efficiency was a primary design focus during the design and implementation of the iScope search algorithms. Experimental results indicate that iScope improves search time and search energy by 4.1X and 3.8X on average, relative to browsing.

References

[1]

"American museum," http://americanhistory.si.edu/.

[2]

"Wikipedia," http://en.wikipedia.org/wiki/Philippe Kahn/.

[3]

"Global mobile forecast to 2012," in Informa Telecomms&Media Report, November 2007.

[4]

R. Datta, et al., "Image retrieval: Ideas, in uences, and trends of the new age," ACM Computing Surveys, 2008.

Digital Library

[5]

C. E. Jacobs, A. Finkelstein, and D. H. Salesin, "Fast multiresolution image querying," in Proceedings of SIGGRAPH 95, August 1995, pp. 277--286.

Digital Library

[6]

Q. Lv, M. Charikar, and K. Li, "Image similarity search with compact data structures," in Proceedings of the 13th ACM Conference on Information and Knowledge Management (CIKM), Washington D.C., USA, November 2004, pp. 208--217.

Digital Library

[7]

D. G. Lowe, "Object recognition from local scale-invariant features," in Proceedings of the International Conference on Computer Vision, September 1999, pp. 1150--1157.

Digital Library

[8]

P. A. Chirita, et al., "Search strategies for scientific collaboration networks," in P2PIR '05: Proceedings of the 2005 ACM workshop on Information retrieval in peer-to-peer networks, Bremen, Germany, 2005, pp. 33--40.

Digital Library

[9]

M. Dalal, "Personalized social&real-time collaborative search," in WWW, 2007, pp. 1285--1286.

Digital Library

[10]

S. Bao, et al., "Optimizing web search using social annotations," in WWW '07: Proceedings of the 16th international conference on World Wide Web, Banff, Alberta, Canada, 2007, pp. 501--510.

Digital Library

[11]

B. E. Kolko, E. J. Rose, and E. J. Johnson, "Communication as information-seeking: the case for mobile social software for developing regions," in WWW, 2007, pp. 863--872.

Digital Library

[12]

P. Heymann, G. Koutrika, and H. Garcia-Molina, "Can social bookmarking improve web search?" in Proceedings of 1st ACM International Conference on Web Search and Data Mining (WSDM'08), 2008.

Digital Library

[13]

SA. Joels, "RFID security and privacy: A research survey," IEEE Journal on Selected Areas in Communications, vol. 24, no. 2, pp. 381--393, Feb. 2006.

Digital Library

[14]

M. J. Beller, L.-F. Chang, and Y. Yacobi, "Privacy and authentication on a portable communications system," IEEE Journal on Selected Areas in Communications, vol. 11, no. 6, pp. 821--829, Aug. 1993.

Digital Library

[15]

F. M. Heikkila, "Encryption: Security considerations for portable media devices," Security&Privacy, IEEE, vol. 5, no. 4, pp. 22--27, July 2007.

Digital Library

[16]

A. Perrig, J. Stankovic, and D. Wagner, "Security in wireless sensor networks," Communications of the ACM, vol. 47, no. 6, pp. 53--57, June 2004.

Digital Library

[17]

E. Berry, et al., "The use of a wearable camera, sensecam, as a pictorial diary to improve autobiographical memory in a patient with limbic encephalitis: A preliminary report."

[18]

C. Y. Kim, et al., "VISCORS: A visual-content recommender for the mobile web," IEEE Intelligent Systems, vol. 19, no. 6, pp. 32--39, 2004.

Digital Library

[19]

T. Yeh, et al., "A picture is worth a thousand keywords: Image-based object search on a mobile platform," in Proc. of the Conference on Human Factors in Computing Systems (CHI), April 2005, pp. 2025--2028.

Digital Library

[20]

I. Ahmad, et al., "Content-based image retrieval on mobile devices," in Proceedings of SPIE (Multimedia on Mobile Devices), vol. 5684, San Jose, CA, USA, January 2005.

[21]

Y. Nam, E. Hwang, and D. Kim, "CLOVER: mobile content-based leaf image retrieval system," in MULTIMEDIA '05: Proceedings of the 13th annual ACM Intl. Conf. on Multimedia, 2005, pp. 215--216.

Digital Library

[22]

M. Jia, et al., "Photo-to-Search: Using camera phones to inquire of the surrounding world," in MDM '06: Proceedings of the 7th International Conference on Mobile Data Management, 2006, pp. 46--46.

Digital Library

[23]

F. James, et al., "Cue ik: interactive concept learning in image search," in Conference on Human Factors in Computing Systems'08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, Apr. 2008, pp. 29--38.

Digital Library

[24]

G. M. Sacco, "Research results in dynamic taxonomy and faceted search systems," in DEXA '07: Proceedings of the 18th International Conference on Database and Expert Systems Applications, 2007, pp. 201--206.

Digital Library

[25]

J. Jeon, V. Lavrenko, and R. Manmatha, "Automatic image annotation and retrieval using cross-media relevance models," in SIGIR, 2003, pp. 119--126.

Digital Library

[26]

T. Yeh, K. Tollmar, and T. Darrell, "Searching the web with mobile images for location recognition," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, July 2004, pp. 76--81.

Digital Library

[27]

D. Cai, et al., "Hierarchical clustering of WWW image search results using visual, textual and link information," in 12th annual ACM international conference on Multimedia.

Digital Library

[28]

N. O'Hare, et al., "Using text search for personal photo collections with the MediAssist system," in SAC '07: Proceedings of the 2007 ACM symposium on Applied computing, 2007, pp. 880--881.

Digital Library

[29]

H. Wang, L.-T. Chia, and S. Liu, "Image retrieval ++-web image retrieval with an enhanced multi-modality ontology," Multimedia Tools Appl., vol. 39, no. 2, pp. 189--215, Nov. 2008.

Digital Library

[30]

K. Veeraraghavan, et al., "quFiles: a unifying abstraction for mobile data management," in HotMobile '08: Proceedings of the 9th workshop on Mobile computing systems and applications, 2008, pp. 65--68.

Digital Library

[31]

T. Sohn, et al., "Experiences with place lab: an open source toolkit for location-aware computing," in ICSE '06: Proceedings of the 28th international conference on Software engineering, 2006, pp. 462--471.

Digital Library

[32]

T. Horozov, N. Narasimhan, and V. Vasudevan, "Using location for personalized POI recommendations in mobile environments," in SAINT '06: Proceedings of the International Symposium on Applications on Internet, 2006, pp. 124--129.

Digital Library

[33]

C. Gurrin, et al., "Mobile access to personal digital photograph archives," in Proc. ACM Int. Conf. Human Computer Interaction with Mobile Devices and Services, 2005, pp. 311--314.

Digital Library

[34]

R. Jesus, et al., "Geographic image retrieval in mobile guides," in GIR '07: Proc. of the 4th ACM workshop on Geographical information retrieval, Nov. 2007, pp. 37--38.

Digital Library

[35]

M. Kahari and D. J. Murphy, "MARA, sensor based augmented reality system for mobile imaging device," in ISMAR, 2006.

[36]

D. Byrne, et al., "Using bluetooth&GPS metadata to measure event similarity in sensecam images," 2007.

[37]

X. Anguera, J. Xu, and N. Oliver, "Multimodal photo annotation and retrieval on a mobile phone," in MIR '08: Proceeding of the 1st ACM international conference on Multimedia information retrieval, 2008, pp. 188--194.

Digital Library

[38]

S. Chakraborty, et al., "On the effectiveness of movement prediction to reduce energy consumption in wireless communication," IEEE Transactions on Mobile Computing, vol. 5, no. 2, pp. 157--169, 2006.

Digital Library

[39]

J. Korhonen and Y. Wang, "Power-efficient streaming for mobile terminals," in NOSSDAV '05: Proceedings of the international workshop on Network and operating systems support for digital audio and video, 2005, pp. 39--44.

Digital Library

[40]

T. Karagiannis, J.-Y. L. Boudec, and M. Vojnović "Power law and exponential decay of inter contact times between mobile devices," in MobiCom '07: Proceedings of the 13th annual ACM international conference on Mobile computing and networking, 2007, pp. 183--194.

Digital Library

[41]

K. Kumar, et al., "Energy conservation by adaptive feature loading for mobile content-based image retrieval," in Proceedings of the thirteenth international symposium on Low power electronics and design, 2008, pp. 153--158.

Digital Library

[42]

C. Yang, M. Dong, and F. Fotouhi, "Semantic feedback for interactive image retrieval," in MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia. ACM, 2005, pp. 415--418.

Digital Library

[43]

S. C. H. Hoi, M. R. Lyu, and R. Jin, "A unified log-based relevance feedback scheme for image retrieval," IEEE Transactions on Knowledge and Data Engineering, vol. 18, pp. 509--524, 2006.

Digital Library

[44]

W. Liu, W. Jiang, and S.-F. Chang, "Relevance aggregation projections for image retrieval," in CIVR '08: Proceedings of the 2008 international conference on Content-based image and video retrieval. New York, NY, USA: ACM, 2008, pp. 119--126.

Digital Library

[45]

S. K. Saha, A. K. Das, and B. Chanda, "Image retrieval based on indexing and relevance feedback," Pattern Recogn. Lett., vol. 28, no. 3, pp. 357--366, 2007.

Digital Library

[46]

A. J. Demers, et al., "The bayou architecture: Support for data sharing among mobile users," in Proceedings of the Workshop on Mobile Computing Systems and Applications, Santa Cruz, CA, USA, December 1994, pp. 2--7.

Digital Library

[47]

D. B. Terry, et al., "Managing update con icts in bayou, a weakly connected replicated storage system," in Proceedings 15th Symposium on Operating Systems Principles (SOSP), Cooper Mountain, CO, USA, December 1995, pp. 172--183.

Digital Library

[48]

S. Sobti, et al., "Segank: A distributed mobile storage system," in Proceedings of the 3rd USENIX Conference on File and Storage Technologies (FAST), March 2004.

Digital Library

[49]

D. Peek and J. Flinn, "Ensemblue: integrating distributed storage and consumer electronics," in OSDI '06: Proceedings of the 7th symposium on Operating systems design and implementation, 2006, pp. 219--232.

Digital Library

[50]

M. A. Smith, D. Davenport, and H. Hwa, "AURA: A mobile platform for object and location annotation," in UbiComp, 2003.

[51]

S. Gaonkar, et al., "Micro-Blog: sharing and querying content through mobile phones and social participation," in MobiSys '08: Proceeding of the 6th international conference on Mobile systems, applications, and services, 2008, pp. 174--186.

Digital Library

[52]

T. Pering, et al., "Face-to-face media sharing using wireless mobile devices," in ISM '05: Proceedings of the Seventh IEEE Int. Symposium on Multimedia, 2005, pp. 269--276.

Digital Library

[53]

T. Pering, et al., "Musicology: Bringing personal music into shared spaces." in MobiQuitous, 2007, pp. 1--8.

Digital Library

[54]

E. Miluzzo, et al., "Sensing meets mobile social networks: the design, implementation and evaluation of the cenceme application," in SenSys '08: Proceedings of the ACM conf. on Embedded network sensor systems, 2008, pp. 337--350.

Digital Library

[55]

D. Dearman, M. Kellar, and K. N. Truong, "An examination of daily information needs and sharing opportunities," in CSCW '08: Proceedings of the ACM 2008 conference on Computer supported cooperative work, 2008, pp. 679--688.

Digital Library

Cited By

Song KNi TSong LXu W(2023)Emma: An accurate, efficient, and multi-modality strategy for autonomous vehicle angle predictionIntelligent and Converged Networks10.23919/ICN.2023.00044:1(41-49)Online publication date: Mar-2023
https://doi.org/10.23919/ICN.2023.0004
Zhao FYan FJin HYang LYu C(2017)Personalized Mobile Searching Approach Based on Combining Content-Based Filtering and Collaborative FilteringIEEE Systems Journal10.1109/JSYST.2015.247299611:1(324-332)Online publication date: Mar-2017
https://doi.org/10.1109/JSYST.2015.2472996
Min WJiang SWang SXu RCao YHerranz LHe Z(2017)A survey on context-aware mobile visual recognitionMultimedia Systems10.1007/s00530-016-0523-823:6(647-665)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1007/s00530-016-0523-8
Show More Cited By

Index Terms

iScope: personalized multi-modality image search for mobile devices
1. Computer systems organization
2. Information systems
  1. Information retrieval

Recommendations

Personalized multi-modality image management and search for mobile devices

Mobile devices are quickly becoming a primary medium for personal information gathering, management, and sharing. Managing personal image data on mobile platforms is a challenging problem due to large data set size, content and context diversity, ...
Decentralized search and retrieval for mobile networks using SMS
WIMOB '12: Proceedings of the 2012 IEEE 8th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob)

This paper describes the iTrust over SMS decentralized search and retrieval system for mobile networks. Any mobile device in the iTrust network can communicate with any other mobile device in the iTrust network to distribute, search for, and retrieve ...
Management of security policies for mobile devices
InfoSecCD '07: Proceedings of the 4th annual conference on Information security curriculum development

This paper discusses management of security policies for mobile devices. The increasing use of mobile devices in the workplace is covered, as well as new software applications that allow employees to use their mobile devices to increase their ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiSys '09: Proceedings of the 7th international conference on Mobile systems, applications, and services

June 2009

370 pages

ISBN:9781605585666

DOI:10.1145/1555816

Co-chair:
Krzysztof Zielinski
AGH University of Science and Technology, Poland
,
General Chair:
Adam Wolisz
Technische Universität Berlin, Germany
,
Program Chairs:
Jason Flinn
University of Michigan, USA
,
Anthony LaMarca
Intel Research Seattle, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 June 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Mobisys '09

Sponsor:

Mobisys '09: The 7th International Conference on Mobile Systems, Applications, and Services

June 22 - 25, 2009

Kraków, Poland

Acceptance Rates

Overall Acceptance Rate 274 of 1,679 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
890
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Song KNi TSong LXu W(2023)Emma: An accurate, efficient, and multi-modality strategy for autonomous vehicle angle predictionIntelligent and Converged Networks10.23919/ICN.2023.00044:1(41-49)Online publication date: Mar-2023
https://doi.org/10.23919/ICN.2023.0004
Zhao FYan FJin HYang LYu C(2017)Personalized Mobile Searching Approach Based on Combining Content-Based Filtering and Collaborative FilteringIEEE Systems Journal10.1109/JSYST.2015.247299611:1(324-332)Online publication date: Mar-2017
https://doi.org/10.1109/JSYST.2015.2472996
Min WJiang SWang SXu RCao YHerranz LHe Z(2017)A survey on context-aware mobile visual recognitionMultimedia Systems10.1007/s00530-016-0523-823:6(647-665)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1007/s00530-016-0523-8
Wu YWang YHu WCao G(2016)SmartPhotoIEEE Transactions on Mobile Computing10.1109/TMC.2015.244437915:5(1249-1263)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1109/TMC.2015.2444379
Wang YHu WWu YCao GWu JCheng XLi XSarkar S(2014)SmartPhotoProceedings of the 15th ACM international symposium on Mobile ad hoc networking and computing10.1145/2632951.2632979(113-122)Online publication date: 11-Aug-2014
https://dl.acm.org/doi/10.1145/2632951.2632979
Yin WMei TChen CLi S(2014)Socialized Mobile Photography: Learning to Photograph With Social Context via Mobile DevicesIEEE Transactions on Multimedia10.1109/TMM.2013.228346816:1(184-200)Online publication date: Jan-2014
https://doi.org/10.1109/TMM.2013.2283468
Xiong HZhang DZhang DGauthier VYang KBecker M(2014)MPaaSInformation Systems Frontiers10.1007/s10796-013-9476-z16:1(59-75)Online publication date: 1-Mar-2014
https://dl.acm.org/doi/10.1007/s10796-013-9476-z
Feng Xia Asabere NAhmed AJing Li Xiangjie Kong (2013)Mobile Multimedia Recommendation in Smart Communities: A SurveyIEEE Access10.1109/ACCESS.2013.22811561(606-624)Online publication date: 2013
https://doi.org/10.1109/ACCESS.2013.2281156
Li KZhu CLv QShang LDick R(2013)Personalized multi-modality image management and search for mobile devicesPersonal and Ubiquitous Computing10.1007/s00779-013-0660-417:8(1817-1834)Online publication date: 1-Dec-2013
https://dl.acm.org/doi/10.1007/s00779-013-0660-4
Kumar KNimmagadda YLu Y(2012)Energy Conservation for Image Retrieval on Mobile SystemsACM Transactions on Embedded Computing Systems10.1145/2345770.234577911:3(1-22)Online publication date: 1-Sep-2012
https://dl.acm.org/doi/10.1145/2345770.2345779
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten