research-article

Lifelong Online Learning from Accumulated Knowledge

Authors:

Changjian Shui,

Christian GagnéAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data, Volume 17, Issue 4

Article No.: 52, Pages 1 - 23

https://doi.org/10.1145/3563947

Published: 24 February 2023 Publication History

Abstract

In this article, we formulate lifelong learning as an online transfer learning procedure over consecutive tasks, where learning a given task depends on the accumulated knowledge. We propose a novel theoretical principled framework, lifelong online learning, where the learning process for each task is in an incremental manner. Specifically, our framework is composed of two-level predictions: the prediction information that is solely from the current task; and the prediction from the knowledge base by previous tasks. Moreover, this article tackled several fundamental challenges: arbitrary or even non-stationary task generation process, an unknown number of instances in each task, and constructing an efficient accumulated knowledge base. Notably, we provide a provable bound of the proposed algorithm, which offers insights on the how the accumulated knowledge improves the predictions. Finally, empirical evaluations on both synthetic and real datasets validate the effectiveness of the proposed algorithm.

References

[1]

Pierre Alquier, The Tien Mai, and Massimiliano Pontil. 2016. Regret bounds for lifelong learning. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 54). Aarti Singh and Jerry Zhu (Eds.). PMLR, 261–269. https://proceedings.mlr.press/v54/alquier17a.html.

[2]

Alejandro Baldominos, Iván Blanco, Antonio José Moreno, Rubén Iturrarte, Óscar Bernárdez, and Carlos Afonso. 2018. Identifying real estate opportunities using machine learning. Applied Sciences 8, 11 (2018), 2321.

[3]

Jonathan Baxter. 2000. A model of inductive bias learning. Journal of Artificial Intelligence Research 12, 1 (2000), 149–198.

[4]

Nicolo Cesa-Bianchi and Gábor Lugosi. 2006. Prediction, Learning, and Games. Cambridge University Press.

[5]

Zhiyuan Chen and Bing Liu. 2016. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning 10, 3 (2016), 1–145.

Digital Library

[6]

Alon Cohen and Shie Mannor. 2017. Online learning with many experts. arXiv preprint arXiv:1702.07870 (2017).

[7]

Giulia Denevi, Carlo Ciliberto, Dimitris Stamos, and Massimiliano Pontil. 2018. Incremental learning-to-learn with statistical guarantees. In Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence (UAI’18). Amir Globerson and Ricardo Silva (Eds.). AUAI Press, Monterey, California, 457–466. http://auai.org/uai2018/proceedings/papers/181.pdf.

[8]

Theodoros Evgeniou and Massimiliano Pontil. 2004. Regularized multi–task learning. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 109–117.

Digital Library

[9]

Geli Fei, Shuai Wang, and Bing Liu. 2016. Learning cumulatively to become more knowledgeable. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1565–1574.

Digital Library

[10]

Shuji Hao, Peiying Hu, Peilin Zhao, Steven C. H. Hoi, and Chunyan Miao. 2018. Online active learning with expert advice. ACM Transactions on Knowledge Discovery from Data 12, 5(2018), 22 pages. DOI:

Digital Library

[11]

Elad Hazan. 2016. Introduction to online convex optimization. Foundations and Trends® in Optimization 2, 3–4 (2016), 157–325.

Digital Library

[12]

Elad Hazan, Tomer Koren, Roi Livni, and Yishay Mansour. 2016. Online learning with low rank experts. In Proceedings of the Conference on Learning Theory. 1096–1114.

[13]

Shuping He, Haiyang Fang, Maoguang Zhang, Fei Liu, and Zhengtao Ding. 2019. Adaptive optimal control for a class of nonlinear systems: The online policy iteration approach. IEEE Transactions on Neural Networks and Learning Systems 31, 2 (2019), 549–558.

[14]

Shuping He, Haiyang Fang, Maoguang Zhang, Fei Liu, Xiaoli Luan, and Zhengdao Ding. 2019. Online policy iterative-based H \(\infty\) optimization algorithm for a class of nonlinear systems. Information Sciences 495 (2019), 1–13.

Digital Library

[15]

Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. 2020. Meta-learning in neural networks: A survey. arXiv:2004.05439. Retrieved from https://arxiv.org/abs/2004.05439.

[16]

Bin Li, Steven C. H. Hoi, Peilin Zhao, and Vivekanand Gopalkrishnan. 2013. Confidence weighted mean reversion strategy for online portfolio selection. ACM Transactions on Knowledge Discovery from Data 7, 1(2013), 38 pages. DOI:

Digital Library

[17]

David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS’17). Curran Associates Inc., Red Hook, NY, 6470–6479.

[18]

Andreas Maurer, Massi Pontil, and Bernardino Romera-Paredes. 2013. Sparse coding for multitask and transfer learning. In Proceedings of the International Conference on Machine Learning. 343–351.

[19]

H. Brendan McMahan. 2017. A survey of algorithms and analysis for adaptive online learning. The Journal of Machine Learning Research 18, 1 (2017), 3117–3166.

Digital Library

[20]

Tom M. Mitchell and Sebastian B. Thrun. 1993. Explanation-based neural network learning for robot control. In Proceedings of the Advances in Neural Information Processing Systems. 287–294.

[21]

Keerthiram Murugesan and Jaime Carbonell. 2017. Multi-task multiple kernel relationship learning. In Proceedings of the 2017 SIAM International Conference on Data Mining. SIAM, 687–695.

[22]

Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22, 10 (2010), 1345–1359.

Digital Library

[23]

Anastasia Pentina and Christoph Lampert. 2014. A PAC-bayesian bound for lifelong learning. In Proceedings of the International Conference on Machine Learning. 991–999.

[24]

Anastasia Pentina and Christoph H. Lampert. 2015. Lifelong learning with non-iid tasks. In Proceedings of the Advances in Neural Information Processing Systems. 1540–1548.

[25]

Anastasia Pentina and Ruth Urner. 2016. Lifelong learning with weighted majority votes. In Proceedings of the Advances in Neural Information Processing Systems. 3612–3620.

[26]

Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016. Progressive neural networks. arXiv preprint arXiv:1606.04671 (2016).

[27]

Paul Ruvolo and Eric Eaton. 2013. Active task selection for lifelong machine learning.Proceedings of the AAAI Conference on Articial Intelligence 27, 1 (2013), 862–868.

[28]

Paul Ruvolo and Eric Eaton. 2013. ELLA: An efficient lifelong learning algorithm. In Proceedings of the International Conference on Machine Learning. 507–515.

[29]

Avishek Saha, Piyush Rai, Hal DaumÃc III, and Suresh Venkatasubramanian. 2011. Online learning of multiple tasks and their relationships. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 15). Geoffrey Gordon, David Dunson, and Miroslav Dudík (Eds.). PMLR, Fort Lauderdale, FL, 643–651. https://proceedings.mlr.press/v15/saha11b.html.

[30]

Shai Shalev-Shwartz and Shai Ben-David. 2014. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press.

[31]

Jun Song, Shuping He, Zhengtao Ding, and Fei Liu. 2016. A new iterative algorithm for solving H \(\infty\) control problem of continuous-time markovian jumping linear systems based on online implementation. International Journal of Robust and Nonlinear Control 26, 17 (2016), 3737–3754.

[32]

Gan Sun, Yang Cong, and Xiaowei Xu. 2018. Active lifelong learning with “watchdog”. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligenc. (AAAI’18/IAAI’18/EAAI’18). AAAI Press, New Orleans, Louisiana, Article 503, 8 pages.

[33]

Joaquin Vanschoren. 2019. Meta-Learning. Springer International Publishing, Cham, 35–61.

[34]

Boyu Wang and Joelle Pineau. 2015. Online boosting algorithms for anytime transfer and multitask learning. In Proceedings of the AAAI Conference on Artificial Intelligence 29, 1 (2015).

[35]

Jun Xiao, Minjuan Wang, Bingqian Jiang, and Junli Li. 2018. A personalized recommendation system with combinational algorithm for online learning. Journal of Ambient Intelligence and Humanized Computing 9, 3 (2018), 667–677.

[36]

Haiqin Yang, Michael R. Lyu, and Irwin King. 2013. Efficient online learning for multitask feature selection. ACM Transactions on Knowledge Discovery from Data 7, 2(2013), 27 pages. DOI:

Digital Library

[37]

Xianchao Zhang, Xiaotong Zhang, and Han Liu. 2015. Smart multitask bregman clustering and multitask kernel clustering. ACM Transactions on Knowledge Discovery from Data 10, 1(2015), 29 pages. DOI:

Digital Library

[38]

Yu Zhang and Qiang Yang. 2017. A survey on multi-task learning. Transactions on Knowledge and Data Engineering 34, 12 (2022), 5586–5609.

[39]

Yu Zhang and Dit-Yan Yeung. 2014. A regularization approach to learning task relationships in multitask learning. ACM Transactions on Knowledge Discovery from Data 8, 3(2014), 31 pages. DOI:

Digital Library

[40]

Peilin Zhao, Steven C. H. Hoi, Jialei Wang, and Bin Li. 2014. Online transfer learning. Artificial Intelligence 216 (2014), 76–102.

Digital Library

[41]

Peilin Zhao, Dayong Wang, Pengcheng Wu, and Steven C. H. Hoi. 2020. A unified framework for sparse online learning. ACM Transactions on Knowledge Discovery from Data 14, 5(2020), 20 pages. DOI:

Digital Library

Cited By

Wang YShang FLei J(2023)Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoTInformation Fusion10.1016/j.inffus.2023.10187199:COnline publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1016/j.inffus.2023.101871

Index Terms

Lifelong Online Learning from Accumulated Knowledge
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Multi-task learning
        Lifelong machine learning
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Online learning theory

Recommendations

Lifelong Machine Learning and Computer Reading the Web
KDD '16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

This tutorial introduces Lifelong Machine Learning (LML) and Machine Reading. The core idea of LML is to learn continuously and accumulate the learned knowledge, and to use the knowledge to help future learning, which is perhaps the hallmark of human ...
PAC Continuous State Online Multitask Reinforcement Learning with Identification
AAMAS '16: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems

One key feature of a general intelligent autonomous agent is to be able to learn from past experience to improve future performance. In this paper we consider how an agent can leverage prior experience from performing reinforcement learning in order to ...
A brief review on multi-task learning

Multi-task learning (MTL), which optimizes multiple related learning tasks at the same time, has been widely used in various applications, including natural language processing, speech recognition, computer vision, multimedia data processing, biomedical ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 17, Issue 4

May 2023

364 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3583065

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 February 2023

Online AM: 17 October 2022

Accepted: 04 September 2022

Revised: 02 July 2022

Received: 13 February 2022

Published in TKDD Volume 17, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Sciences and Engineering Research Council of Canada (NSERC), Discovery Grants program, the Science and Technology Development Fund, Macau SAR
University of Macau

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
343
Total Downloads

Downloads (Last 12 months)160
Downloads (Last 6 weeks)17

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang YShang FLei J(2023)Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoTInformation Fusion10.1016/j.inffus.2023.10187199:COnline publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1016/j.inffus.2023.101871

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents