research-article

On Scalability of Association-rule-based Recommendation: A Unified Distributed-computing Framework

Authors:

Yong GeAuthors Info & Claims

ACM Transactions on the Web (TWEB), Volume 14, Issue 3

Article No.: 13, Pages 1 - 21

https://doi.org/10.1145/3398202

Published: 21 June 2020 Publication History

Abstract

The association-rule-based approach is one of the most common technologies for building recommender systems and it has been extensively adopted for commercial use. A variety of techniques, mainly including eligible rule selection and multiple rules combination, have been developed to create effective recommendation. Unfortunately, little attention has been paid to the scalability concern of rule-based recommendation methods. However, the computational complexity of rule-based methods shall increase drastically with the growth of both online customers and rules, which are usually several millions in typical e-commerce platforms. Moreover, the dynamic change of users’ actions requires rule-based methods make recommendations in nearly real-time, which further highlights the scalability issue of rule-based recommender systems. In this article, we present a distributed framework that can scale different association-rule-based recommendation methods in a unified way. Specifically, based on the summarization of existing rule-based approaches, a generic tree-type structure is defined to store separate kinds of patterns, and an efficient algorithm is designed for mining eligible patterns along with computing recommendation scores. To handle the ever-increasing number of online customers, a distributed framework is proposed, where two load-balanced strategies for partitioning tree are put forward to fit sparse and dense data, respectively. Extensive experiments on five real-life data sets demonstrate that the efficiency of association-rule-based recommender systems can be significantly improved by the proposed framework.

References

[1]

Gediminas Adomavicius and Alexander Tuzhilin. 2001. Expert-driven validation of rule-based user models in personalization applications. Data Min. Knowl. Discov. 5, 1 (2001), 33--58.

Digital Library

[2]

Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17, 6 (2005), 734--749.

Digital Library

[3]

Gediminas Adomavicius and Alexander Tuzhilin. 2007. Validation sequence optimization: A theoretical approach. INFORMS J. Comput. 19, 2 (2007), 185--200.

Digital Library

[4]

Charu C. Aggarwal. 2016. Recommender Systems. Springer.

Digital Library

[5]

Rakesh Agrawal, Tomasz Imielinski, and Arun Swami. 1993. Mining association rules between sets of items in large databases. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM Press, New York, 207--216.

Digital Library

[6]

Gregory Buehrer, Srinivasan Parthasarathy, and Amol Ghoting. 2006. Out-of-core frequent pattern mining on a commodity PC. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 86--95.

Digital Library

[7]

B. Barla Cambazoglu, Emre Varol, Enver Kayaaslan, Cevdet Aykanat, and Ricardo Baeza-Yates. 2010. Query forwarding in geographically distributed search engines. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 90--97.

Digital Library

[8]

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2001. Introduction to Algorithms. MIT Press.

[9]

James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, and Dasarathi Sampath. 2010. The YouTube video recommendation system. In Proceedings of the ACM Conference on Recommender Systems (RecSys’10). 293--296.

Digital Library

[10]

Farzad Eskandanian, Bamshad Mobasher, and Robin Burke. 2017. A clustering approach for personalizing diversity in collaborative recommender systems. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP’17). 280--284.

Digital Library

[11]

Rana Forsati and Mohammad Reza Meybodi. 2010. Effective page recommendation algorithms based on distributed learning automata and weighted association rules. Expert Syst. Appl. 37, 2 (2010), 1316--1330.

Digital Library

[12]

Gustavo Machado Campagnani Gama, W. Meira, Márcio L. B. Carvalho, Dorgival O. Guedes, and Virgílio A. F. Almeida. 2001. Resource placement in distributed e-commerce servers. In Proceedings of the IEEE Global Telecommunications Conference (GLOBECOM’01), Vol. 3. IEEE, 1677--1682.

[13]

Abhijeet Ghoshal, Syam Menon, and Sumit Sarkar. 2015. Recommendations using information from multiple association rules: A probabilistic approach. Information Syst. Res. 26, 3 (2015), 532--551.

Digital Library

[14]

Abhijeet Ghoshal and Sumit Sarkar. 2014. Association rules for recommendations with multiple items. Informs J. Comput. 26, 3 (2014), 433--448.

[15]

R. L. Graham. 1969. Bounds on multiprocessing timing anomalies. SIAM J. Appl. Math. 17, 2 (1969), 416--429.

Digital Library

[16]

Gösta Grahne and Jianfei Zhu. 2004. Mining frequent itemsets from secondary memory. In Proceedings of the International Conference on Data Mining (ICDM’04). 91--98.

[17]

Grahne Grahne and Jianfei Zhu. 2005. Fast algorithms for frequent itemset mining using FP-trees. IEEE Trans. Knowl. Data Eng. 17, 10 (2005), 1347--1362.

Digital Library

[18]

Guibing Guo, Jie Zhang, and Neil Yorke-Smith. 2016. A novel evidence-based Bayesian similarity measure for recommender systems. ACM Trans. Web 10, 2 (2016), 8.

Digital Library

[19]

Jiawei Han, Jian Pei, and Yiwen Yin. 2000. Mining frequent patterns without candidate generation. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM Press, New York, 1--12.

Digital Library

[20]

Negar Hariri, Bamshad Mobasher, and Robin Burke. 2012. Context-aware music recommendation based on latent topic sequential patterns. In Proceedings of the 6th ACM Conference on Recommender Systems. ACM, 131--138.

Digital Library

[21]

Choonho Kim and Juntae Kim. 2003. A recommendation algorithm using multi-level association rules. In Proceedings of the IEEE / WIC International Conference on Web Intelligence (WI’03). 524--527.

[22]

Carson K. Leung, Fan Jiang, and Adam GM Pazdor. 2017. Bitwise parallel association rule mining for web page recommendation. In Proceedings of the International Conference on Web Intelligence. ACM, 662--669.

Digital Library

[23]

Cane Wing-ki Leung, Stephen Chi-fai Chan, and Fu-lai Chung. 2006. A collaborative filtering framework based on fuzzy association rules and multiple-level similarity. Knowl. Info. Syst. 10, 3 (2006), 357--381.

Digital Library

[24]

Haoyuan Li, Yi Wang, Dong Zhang, Ming Zhang, and Edward Y. Chang. 2008. PFP: Parallel fp-growth for query recommendation. In Proceedings of the ACM Conference on Recommender Systems (RecSys’08). 107--114.

[25]

Wenmin Li, Jiawei Han, and Jian Pei. 2001. CMAR: Accurate and efficient classification based on multiple class-association rules. In Proceedings of the IEEE International Conference on Data Mining. 19--21.

[26]

Weiyang Lin, Sergio A. Alvarez, and Carolina Ruiz. 2002. Efficient adaptive-support association rule mining for recommender systems. Data Mining Knowl. Discov. 6, 1 (2002), 83--105.

Digital Library

[27]

Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 7, 1 (2003), 76--80.

Digital Library

[28]

Duen-Ren Liu, Chin-Hui Lai, and Wang-Jung Lee. 2009. A hybrid of sequential rules and collaborative filtering for product recommendation. Info. Sci. 179, 20 (2009), 3505--3519.

[29]

Xin Luo, Mengchu Zhou, Yunni Xia, and Qingsheng Zhu. 2014. An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems. IEEE Trans. Industr. Inform. 10, 2 (2014), 1273--1284.

[30]

Julian J. McAuley and Jure Leskovec. 2013. Hidden factors and hidden topics: Understanding rating dimensions with review text. In Proceedings of the 7th ACM Conference on Recommender Systems (RecSys’13). 165--172.

[31]

Harshitha Menon and Laxmikant V. Kalé. 2013. A distributed dynamic load balancer for iterative applications. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC’13). 15.

[32]

Bamshad Mobasher, Honghua Dai, Tao Luo, and Miki Nakagawa. 2001. Effective personalization based on association rule discovery from web usage data. In Proceedings of the 3rd International Workshop on Web Information and Data Management (WIDM’01). 9--15.

Digital Library

[33]

Bamshad Mobasher, Honghua Dai, Tao Luo, and Miki Nakagawa. 2002. Discovery and evaluation of aggregate usage profiles for web personalization. Data Mining Knowl. Discov. 6, 1 (2002), 61--82.

Digital Library

[34]

Kaweh Djafari Naini, Ismail Sengor Altingovde, and Wolf Siberski. 2016. Scalable and efficient web search result diversification. ACM Trans. Web 10, 3 (2016), 15.

Digital Library

[35]

Miki Nakagawa and Bamshad Mobasher. 2003. A hybrid web personalization model based on site connectivity. In Proceedings of the SIGKDD Workshop on Web Mining and Web Usage Analysis (WebKDD’03). 59--70.

[36]

Sergio Oramas, Vito Claudio Ostuni, Tommaso Di Noia, Xavier Serra, and Eugenio Di Sciascio. 2016. Sound and music recommendation with knowledge graphs. ACM Trans. Intell. Syst. Technol. 8, 2 (2016), 21.

[37]

Srinivasan Parthasarathy, Mohammed Javeed Zaki, Mitsunori Ogihara, and Wei Li. 2001. Parallel data mining for association rules on shared-memory systems. Knowl. Info. Syst. 3, 1 (2001), 1--29.

Digital Library

[38]

David Patterson and John L. Hennessy. 2012. Computer Architecture: A Quantitative Approach. Elsevier.

[39]

Zujie Ren, Jian Wan, Weisong Shi, Xianghua Xu, and Min Zhou. 2014. Workload analysis, implications, and optimization on a production hadoop cluster: A case study on Taobao. IEEE Trans. Services Comput. 7, 2 (2014), 307--321.

[40]

Cynthia Rudin, Benjamin Letham, Ansaf Salleb-Aouissi, Eugene Kogan, and David Madigan. 2011. Sequential event prediction with association rules. Omnipress 19 (2011), 615--634.

[41]

J. J. Sandvig, Bamshad Mobasher, and Robin Burke. 2007. Robustness of collaborative recommendation based on association rule mining. In Proceedings of the ACM Conference on Recommender Systems. 105--112.

Digital Library

[42]

Evan R. Sparks, Ameet Talwalkar, Valton Smith, Jey Kottalam, Xinghao Pan, Jose Gonzalez, Michael J. Franklin, Michael I. Jordan, and Tim Kraska. 2013. MLI: An API for distributed machine learning. In Proceedings of the IEEE 13th International Conference on Data Mining (ICDM’13). IEEE, 1187--1192.

[43]

Myra Spiliopoulou and Lukas C. Faulstich. 1998. WUM: A web utilization miner. In Proceedings of the EDBT Workshop (WebDB’98). Springer Verlag, 109--115.

[44]

Jiaxi Tang and Ke Wang. 2018. Personalized Top-N sequential recommendation via convolutional sequence embedding. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM’18). 565--573.

Digital Library

[45]

Fatemeh Vahedian, Robin Burke, and Bamshad Mobasher. 2017. Multirelational recommendation in heterogeneous networks. ACM Trans. Web 11, 3 (2017), 15:1--15:34.

Digital Library

[46]

Feng Hsu Wang and Hsiu Mei Shao. 2004. Effective personalized recommendation based on time-framed navigation clustering and association mining. Expert Syst. Appl. 27, 3 (2004), 365--377.

[47]

Yaqiong Wang, Junjie Wu, Zhiang Wu, Hua Yuan, and Xu Zhang. 2014. Popular items or niche items: Flexible recommendation using cosine patterns. In Proceedings of the IEEE International Conference on Data Mining Workshops. 205--212.

[48]

Geoffrey I. Webb. 2000. Efficient search for association rules. In Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 99--107.

Digital Library

[49]

Kasun Wickramaratna, Miroslav Kubat, and Kamal Premaratne. 2009. Predicting missing items in shopping carts. IEEE Trans. Knowl. Data Eng. 21, 7 (2009), 985--998.

Digital Library

[50]

Le Wu, Qi Liu, Enhong Chen, Nicholas Jing Yuan, Guangming Guo, and Xing Xie. 2016. Relevance meets coverage: A unified framework to generate diversified recommendations. ACM Trans. Intell. Syst. Technol. 7, 3 (2016), 39:1--39:30.

Digital Library

[51]

Hui Xiong, Pang-Ning Tan, and Vipin Kumar. 2006. Hyperclique pattern discovery. Data Mining Knowl. Discov. J. 13, 2 (2006), 219--242.

Digital Library

[52]

Osmar R. Zaïane. 2002. Building a recommender agent for e-learning systems. In Proceedings of the IEEE International Conference on Computers in Education. IEEE, 55--59.

[53]

Heng-Ru Zhang, Fan Min, and Bing Shi. 2017. Regression-based three-way recommendation. Info. Sci. 378 (2017), 444--461.

[54]

Mei Zheng, Fan Min, Heng-Ru Zhang, and Wen-Bin Chen. 2016. Fast recommendations with the M-distance. IEEE Access 4 (2016), 1464--1468.

[55]

Guixiang Zhu, Jie Cao, Changsheng Li, and Zhiang Wu. 2017. A recommendation engine for travel products based on topic sequential patterns. Multimedia Tools Appl. 76, 16 (2017), 17595--17612.

Digital Library

[56]

Jiaqi Zhu, Kaijun Wang, Yunkun Wu, Zhongyi Hu, and Hongan Wang. 2016. Mining user-aware rare sequential topic patterns in document streams. IEEE Trans. Knowl. Data Eng. 28, 7 (2016), 1790--1804.

Cited By

Gao MWang JLiu O(2024)Is UGC sentiment helpful for recommendation? An application of sentiment-based recommendation modelIndustrial Management & Data Systems10.1108/IMDS-05-2023-0335Online publication date: 16-Feb-2024
https://doi.org/10.1108/IMDS-05-2023-0335
Dong RSun LCai ZHeidari ALiu LChen H(2024)An advanced kernel search optimization for dynamic economic emission dispatch with new energy sourcesInternational Journal of Electrical Power & Energy Systems10.1016/j.ijepes.2024.110085160(110085)Online publication date: Sep-2024
https://doi.org/10.1016/j.ijepes.2024.110085
Yang XQuan WZhou JZhang OWang XLiu C(2024)A new machine learning model to predict the prognosis of cardiogenic brain infarctionComputers in Biology and Medicine10.1016/j.compbiomed.2024.108600178(108600)Online publication date: Aug-2024
https://doi.org/10.1016/j.compbiomed.2024.108600
Show More Cited By

Index Terms

On Scalability of Association-rule-based Recommendation: A Unified Distributed-computing Framework
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
2. Information systems
  1. Information systems applications
    1. Data mining
      1. Association rules
  2. World Wide Web
    1. Web applications
      1. Electronic commerce
    2. Web searching and information discovery
      1. Personalization

Recommendations

Association rule and quantitative association rule mining among infrequent items
MDM '07: Proceedings of the 8th international workshop on Multimedia data mining: (associated with the ACM SIGKDD 2007)

Association rule mining among frequent items has been extensively studied in data mining research. However, in the recent years, there is an increasing demand of mining the infrequent items (such as rare but expensive items). Since exploring interesting ...
Non-Redundant Sequential Association Rule Mining and Application in Recommender Systems
WI-IAT '10: Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 03

Many modern recommender systems are not suitable for recommending infrequently purchased products such as cars due to lack of user rating data to infrequently purchased products. A big challenge for recommending infrequently purchased products is the ...
Hiding collaborative recommendation association rules

The concept of Privacy-Preserving has recently been proposed in response to the concerns of preserving personal or sensible information derived from data mining algorithms. For example, through data mining, sensible information such as private ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on the Web

ACM Transactions on the Web Volume 14, Issue 3

August 2020

126 pages

ISSN:1559-1131

EISSN:1559-114X

DOI:10.1145/3398019

Editor:
Brian D. Davison
Lehigh University, USA

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2020

Online AM: 07 May 2020

Accepted: 01 May 2020

Revised: 01 April 2019

Received: 01 December 2017

Published in TWEB Volume 14, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Industry Projects in Jiangsu S8T Pillar Program
National Natural Science Foundation of China (NSFC)
National Key Research and Development Program of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
534
Total Downloads

Downloads (Last 12 months)67
Downloads (Last 6 weeks)3

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gao MWang JLiu O(2024)Is UGC sentiment helpful for recommendation? An application of sentiment-based recommendation modelIndustrial Management & Data Systems10.1108/IMDS-05-2023-0335Online publication date: 16-Feb-2024
https://doi.org/10.1108/IMDS-05-2023-0335
Dong RSun LCai ZHeidari ALiu LChen H(2024)An advanced kernel search optimization for dynamic economic emission dispatch with new energy sourcesInternational Journal of Electrical Power & Energy Systems10.1016/j.ijepes.2024.110085160(110085)Online publication date: Sep-2024
https://doi.org/10.1016/j.ijepes.2024.110085
Yang XQuan WZhou JZhang OWang XLiu C(2024)A new machine learning model to predict the prognosis of cardiogenic brain infarctionComputers in Biology and Medicine10.1016/j.compbiomed.2024.108600178(108600)Online publication date: Aug-2024
https://doi.org/10.1016/j.compbiomed.2024.108600
Chen XZhao DJi HChen YLi YZuo Z(2024)Predictive modeling for early detection of biliary atresia in infants with cholestasis: Insights from a machine learning studyComputers in Biology and Medicine10.1016/j.compbiomed.2024.108439174(108439)Online publication date: May-2024
https://doi.org/10.1016/j.compbiomed.2024.108439
Huang WJi YZhou XHe BTan K(2023)A Design Space Exploration and Evaluation for Main-Memory Hash Joins in Storage Class MemoryProceedings of the VLDB Endowment10.14778/3583140.358314416:6(1249-1263)Online publication date: 20-Apr-2023
https://dl.acm.org/doi/10.14778/3583140.3583144
Nguyen Thanh TQuach NNguyen THuynh TVu VNguyen PJo JNguyen Q(2023)Poisoning GNN-based Recommender Systems with Generative Surrogate-based AttacksACM Transactions on Information Systems10.1145/356742041:3(1-24)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1145/3567420
Costache RArabameri ACostache ICrăciun AIslam AAbba SSahana MPandey MTin TPham B(2023)Flood hazard potential evaluation using decision tree state‐of‐the‐art modelsRisk Analysis10.1111/risa.14179Online publication date: 25-Jun-2023
https://doi.org/10.1111/risa.14179
Yang Y(2023)University Teaching Evaluation Based on APriori Algorithm and Data Mining2023 International Conference on Network, Multimedia and Information Technology (NMITCON)10.1109/NMITCON58196.2023.10276265(1-8)Online publication date: 1-Sep-2023
https://doi.org/10.1109/NMITCON58196.2023.10276265
Sun XZhou JLiu LWei W(2023)Explicit time embedding based cascade attention network for information popularity predictionInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10327860:3Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1016/j.ipm.2023.103278
Wu SHeidari AZhang SKuang FChen H(2023)Gaussian bare-bone slime mould algorithm: performance optimization and case studies on truss structuresArtificial Intelligence Review10.1007/s10462-022-10370-756:9(9051-9087)Online publication date: 20-Jan-2023
https://dl.acm.org/doi/10.1007/s10462-022-10370-7
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents