research-article

Photo2Trip: Exploiting Visual Contents in Geo-tagged Photos for Personalized Tour Recommendation

Authors:

Victor S. Sheng,

Hui XiongAuthors Info & Claims

MM '17: Proceedings of the 25th ACM international conference on Multimedia

Pages 916 - 924

https://doi.org/10.1145/3123266.3123336

Published: 19 October 2017 Publication History

Abstract

Recently accumulated massive amounts of geo-tagged photos provide an excellent opportunity to understand human behaviors and can be used for personalized tour recommendation. However, no existing work has considered the visual content information in these photos for tour recommendation. We believe the visual features of photos provide valuable information on measuring user / Point-of-Interest (POI) similarities, which is challenging due to data sparsity. To this end, in this paper, we propose a visual feature enhanced tour recommender system, named 'Photo2Trip', to utilize the visual contents and collaborative filtering models for recommendation. Specifically, we first extract various visual features from photos taken by tourists. Then, we propose a Visual-enhanced Probabilistic Matrix Factorization model (VPMF), which integrates visual features into the collaborative filtering model, to learn user interests by leveraging the historical travel records. Moreover, user interests together with trip constraints are formalized to an optimization problem for trip planning. Finally, the experimental results on real-world data show that our proposed visual-enhanced personalized tour recommendation method outperforms other benchmark methods in terms of recommendation accuracy. The results also show that visual features are effective on alleviating the data sparsity and cold start problems on personalized tour recommendation.

References

[1]

Ryan Prescott Adams, George E. Dahl, and Iain Murray. 2010. Incorporating Side Information in Probabilistic Matrix Factorization with Gaussian Processes. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI'10). AUAI Press, 1--9.

Digital Library

[2]

Yuki Arase, Xing Xie, Takahiro Hara, and Shojiro Nishio. 2010. Mining people's trips from large scale geo-tagged photos. In Proceedings of the 18th ACM international conference on Multimedia. ACM, 133--142.

Digital Library

[3]

Michel Berkelaar, Kjell Eikland, Peter Notebaert, and others. 2004. lpsolve: Open source (mixed-integer) linear programming system. Eindhoven U. of Technology (2004).

[4]

Igo Brilhante, Jose Antonio Macedo, Franco Maria Nardini, Raffaele Perego, and Chiara Renso. 2013. Where shall we go today?: planning touristic tours with tripbuilder. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. ACM, 757--762.

Digital Library

[5]

Igo Ramalho Brilhante, Jose Antonio Macedo, Franco Maria Nardini, Raffaele Perego, and Chiara Renso. 2015. On planning sightseeing tours with TripBuilder. Information Processing & Management 51, 2 (2015), 1--15.

[6]

An-Jung Cheng, Yan-Ying Chen, Yen-Ta Huang, Winston H Hsu, and Hong-Yuan Mark Liao. 2011. Personalized travel recommendation by mining people attributes from community-contributed photos. In Proceedings of the 19th ACM international conference on Multimedia. ACM, 83--92.

Digital Library

[7]

Chen Cheng, Haiqin Yang, Irwin King, and Michael R Lyu. 2012. Fused Matrix Factorization with Geographical and Social Influence in Location-Based Social Networks. In Aaai, Vol. 12. 1.

Digital Library

[8]

Gabriella Csurka, Christopher Dance, Lixin Fan, Jutta Willamowski, and Cédric Bray. 2004. Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV, Vol. 1. Prague, 1--22.

[9]

Dominique Feillet, Pierre Dejax, and Michel Gendreau. 2005. Traveling salesman problems with profits. Transportation science 39, 2 (2005), 188--205.

Digital Library

[10]

Huiji Gao, Jiliang Tang, Xia Hu, and Huan Liu. 2013. Exploring temporal effects for location recommendation on location-based social networks. In Proceedings of the 7th ACM conference on Recommender systems. ACM, 93--100.

Digital Library

[11]

Damianos Gavalas, Charalampos Konstantopoulos, Konstantinos Mastakas, and Grammati Pantziou. 2014. A survey on algorithmic approaches for solving tourist trip design problems. Journal of Heuristics 20, 3 (2014), 291--328.

Digital Library

[12]

Yong Ge, Qi Liu, Hui Xiong, Alexander Tuzhilin, and Jian Chen. 2011. Cost-aware travel tour recommendation. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 983--991.

Digital Library

[13]

Quanquan Gu, Jie Zhou, and Chris Ding. 2010. Collaborative filtering: Weighted nonnegative matrix factorization incorporating user and item graphs. In Proceedings of the 2010 SIAM International Conference on Data Mining. SIAM, 199--210.

[14]

Ron Kohavi and others. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In IJCAI, Vol. 14. Stanford, CA, 1137--1145.

Digital Library

[15]

Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 426--434.

Digital Library

[16]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization tech- niques for recommender systems. Computer 42, 8 (2009), 30--37.

Digital Library

[17]

Kwan Hui Lim. 2015. Recommending tours and places-of-interest based on user interests from geo-tagged photos. In Proceedings of the 2015 ACM SIGMOD on PhD Symposium. ACM, 33--38.

Digital Library

[18]

Kwan Hui Lim, Jeffrey Chan, Christopher Leckie, and Shanika Karunasekera. 2015. Personalized Tour Recommendation Based on User Interests and Points of Interest Visit Durations. In IJCAI. 1778--1784.

Digital Library

[19]

Yanchi Liu, Chuanren Liu, Bin Liu, Meng Qu, and Hui Xiong. 2016. Unified Point- of-Interest Recommendation with Temporal Interval Assessment. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1015--1024.

Digital Library

[20]

Yanchi Liu, Chuanren Liu, Nicholas Jing Yuan, Lian Duan, Yanjie Fu, Hui Xiong, Songhua Xu, and Junjie Wu. 2014. Exploiting heterogeneous human mobility patterns for intelligent bus routing. In Data Mining (ICDM), 2014 IEEE International Conference on. IEEE, 360--369.

Digital Library

[21]

David G Lowe. 1999. Object recognition from local scale-invariant features. In Computer vision, 1999. The proceedings of the seventh IEEE international conference on, Vol. 2. Ieee, 1150--1157.

Digital Library

[22]

Xin Lu, Changhu Wang, Jiang-Ming Yang, Yanwei Pang, and Lei Zhang. 2010. Photo2trip: generating travel routes from geo-tagged photos for trip planning. In Proceedings of the 18th ACM international conference on Multimedia. ACM, 143--152.

Digital Library

[23]

Ruslan Salakhutdinov and Andriy Mnih. 2007. Probabilistic Matrix Factorization. In Nips, Vol. 1. 1257--1264.

Digital Library

[24]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[25]

Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 2015. The new data and new challenges in multimedia research. arXiv preprint arXiv:1503.01817 1, 8 (2015).

[26]

Xinxi Wang and Ye Wang. 2014. Improving content-based and hybrid music recommendation using deep learning. In Proceedings of the 22nd ACM international conference on Multimedia. ACM, 627--636.

Digital Library

[27]

Le Wu, Enhong Chen, Qi Liu, Linli Xu, Tengfei Bao, and Lei Zhang. 2012. Lever- aging tagging for neighborhood-aware probabilistic matrix factorization. In Proceedings of the 21st ACM international conference on Information and knowledge management. ACM, 1854--1858.

Digital Library

[28]

Mao Ye, Peifeng Yin, and Wang-Chien Lee. 2010. Location recommendation for location-based social networks. In Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems. ACM, 458--461.

Digital Library

[29]

Mao Ye, Peifeng Yin, Wang-Chien Lee, and Dik-Lun Lee. 2011. Exploiting geographical influence for collaborative point-of-interest recommendation. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. ACM, 325--334.

Digital Library

[30]

Quan Yuan, Gao Cong, Zongyang Ma, Aixin Sun, and Nadia Magnenat Thalmann. 2013. Time-aware point-of-interest recommendation. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. ACM, 363--372.

Digital Library

Cited By

Zou XYan YHao XHu YWen HLiu EZhang JLi YLi TZheng YLiang Y(2024)Deep learning for cross-domain data fusion in urban computing: Taxonomy, advances, and outlookInformation Fusion10.1016/j.inffus.2024.102606(102606)Online publication date: Aug-2024
https://doi.org/10.1016/j.inffus.2024.102606
Chen LLyu DYu SChen G(2023)Multi-Level Visual Similarity Based Personalized Tourist Attraction Recommendation Using Geo-Tagged PhotosACM Transactions on Knowledge Discovery from Data10.1145/358201517:7(1-18)Online publication date: 6-Apr-2023
https://dl.acm.org/doi/10.1145/3582015
Deng LZhao YSun HYang CXie JZheng K(2023)Fusing Local and Global Mobility Patterns for Trajectory RecoveryDatabase Systems for Advanced Applications10.1007/978-3-031-30637-2_29(448-463)Online publication date: 14-Apr-2023
https://doi.org/10.1007/978-3-031-30637-2_29
Show More Cited By

Index Terms

Photo2Trip: Exploiting Visual Contents in Geo-tagged Photos for Personalized Tour Recommendation
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems
  2. Information systems applications
    1. Data mining

Recommendations

Personalized trip recommendation for tourists based on user interests, points of interest visit durations and visit recency

Tour recommendation and itinerary planning are challenging tasks for tourists, due to their need to select points of interest (POI) to visit in unfamiliar cities and to select POIs that align with their interest preferences and trip constraints. We ...
Recommending Tours and Places-of-Interest based on User Interests from Geo-tagged Photos
SIGMOD '15 PhD Symposium: Proceedings of the 2015 ACM SIGMOD on PhD Symposium

Photo sharing sites like Flickr and Instagram have grown increasingly popular in recent years, resulting in a large amount of uploaded photos. In addition, these photos contain useful meta-data such as the taken time and geo-location. Using such geo-...
User preference representation based on psychometric models
ADC '11: Proceedings of the Twenty-Second Australasian Database Conference - Volume 115

Neighbourhood-based collaborative filtering is one of the most popular recommendation techniques, and has been applied successfully in various fields. User ratings are often used by neighbourhood-based collaborative filtering to compute the similarity ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '17: Proceedings of the 25th ACM international conference on Multimedia

October 2017

2028 pages

ISBN:9781450349062

DOI:10.1145/3123266

General Chairs:
Qiong Liu
FXPAL, USA
,
Rainer Lienhart
Universität Augsburg, Germany
,
Haohong Wang
TCL America, USA
,
Program Chairs:
Sheng-Wei "Kuan-Ta" Chen
Academia Sinica, Taiwan
,
Susanne Boll
University of Oldenburg, Germany
,
Phoebe Chen
La Trobe University, Australia
,
Gerald Friedland
Lawrence Livermore National Lab, USA
,
Jia Li
Google, USA
,
Shuicheng Yan
Qihoo 360, China

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

the Natural Science Foundation of China

Conference

MM '17

Sponsor:

SIGMM

MM '17: ACM Multimedia Conference

October 23 - 27, 2017

California, Mountain View, USA

Acceptance Rates

MM '17 Paper Acceptance Rate 189 of 684 submissions, 28%;

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
391
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)1

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zou XYan YHao XHu YWen HLiu EZhang JLi YLi TZheng YLiang Y(2024)Deep learning for cross-domain data fusion in urban computing: Taxonomy, advances, and outlookInformation Fusion10.1016/j.inffus.2024.102606(102606)Online publication date: Aug-2024
https://doi.org/10.1016/j.inffus.2024.102606
Chen LLyu DYu SChen G(2023)Multi-Level Visual Similarity Based Personalized Tourist Attraction Recommendation Using Geo-Tagged PhotosACM Transactions on Knowledge Discovery from Data10.1145/358201517:7(1-18)Online publication date: 6-Apr-2023
https://dl.acm.org/doi/10.1145/3582015
Deng LZhao YSun HYang CXie JZheng K(2023)Fusing Local and Global Mobility Patterns for Trajectory RecoveryDatabase Systems for Advanced Applications10.1007/978-3-031-30637-2_29(448-463)Online publication date: 14-Apr-2023
https://doi.org/10.1007/978-3-031-30637-2_29
Wu YLi KZhao GQian X(2022)Personalized Long- and Short-term Preference Learning for Next POI RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2020.300253134:4(1944-1957)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TKDE.2020.3002531
Renjith SSreekumar AJathavedan M(2022)An extensive study on the evolution of context-aware personalized travel recommender systemsInformation Processing and Management: an International Journal10.1016/j.ipm.2019.10207857:1Online publication date: 21-Apr-2022
https://dl.acm.org/doi/10.1016/j.ipm.2019.102078
Gutierres FGomes P(2021)The Integrated Tourism Analysis Platform (ITAP) for Tourism Destination ManagementEncyclopedia of Organizational Knowledge, Administration, and Technology10.4018/978-1-7998-3473-1.ch113(1651-1660)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-3473-1.ch113
Zhou FWang PXu XTai WTrajcevski G(2021)Contrastive Trajectory Learning for Tour RecommendationACM Transactions on Intelligent Systems and Technology10.1145/346233113:1(1-25)Online publication date: 29-Nov-2021
https://dl.acm.org/doi/10.1145/3462331
Ameen TChen LXu ZLyu DShi H(2020)A Convolutional Neural Network and Matrix Factorization-Based Travel Location Recommendation Method Using Community-Contributed Geotagged PhotosISPRS International Journal of Geo-Information10.3390/ijgi90804649:8(464)Online publication date: 22-Jul-2020
https://doi.org/10.3390/ijgi9080464
Liu JZhao PZhuang FLiu YSheng VXu JZhou XXiong H(2020)Exploiting Aesthetic Preference in Deep Cross Networks for Cross-domain RecommendationProceedings of The Web Conference 202010.1145/3366423.3380036(2768-2774)Online publication date: 20-Apr-2020
https://dl.acm.org/doi/10.1145/3366423.3380036
Tang JDu XHe XYuan FTian QChua T(2020)Adversarial Training Towards Robust Multimedia Recommender SystemIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2019.289363832:5(855-867)Online publication date: 1-May-2020
https://doi.org/10.1109/TKDE.2019.2893638
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents