research-article

Enhanced Knowledge-Leverage-Based TSK Fuzzy System Modeling for Inductive Transfer Learning

Authors:

Hisao Ishibuchi,

Shitong WangAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 8, Issue 1

Article No.: 11, Pages 1 - 21

https://doi.org/10.1145/2903725

Published: 25 July 2016 Publication History

Abstract

The knowledge-leverage-based Takagi--Sugeno--Kang fuzzy system (KL-TSK-FS) modeling method has shown promising performance for fuzzy modeling tasks where transfer learning is required. However, the knowledge-leverage mechanism of the KL-TSK-FS can be further improved. This is because available training data in the target domain are not utilized for the learning of antecedents and the knowledge transfer mechanism from a source domain to the target domain is still too simple for the learning of consequents when a Takagi--Sugeno--Kang fuzzy system (TSK-FS) model is trained in the target domain. The proposed method, that is, the enhanced KL-TSK-FS (EKL-TSK-FS), has two knowledge-leverage strategies for enhancing the parameter learning of the TSK-FS model for the target domain using available information from the source domain. One strategy is used for the learning of antecedent parameters, while the other is for consequent parameters. It is demonstrated that the proposed EKL-TSK-FS has higher transfer learning abilities than the KL-TSK-FS. In addition, the EKL-TSK-FS has been further extended for the scene of the multisource domain.

Supplementary Material

a11-deng-apndx.pdf (deng.zip)

Supplemental movie, appendix, image and software files for, Enhanced Knowledge-Leverage-Based TSK Fuzzy System Modeling for Inductive Transfer Learning

Download
185.19 KB

References

[1]

M. F. Azeem, M. Hanmandlu, and N. Ahmad. 2000. Generalization of adaptive neural-fuzzy inference systems. IEEE Trans. Neural Netw. 11, 6, 1332--1346.

Digital Library

[2]

J. C. Bezdek, J. Keller, and R. Krishnapuram. 1999. Fuzzy Models and Algorithms for Pattern Recognition and Image Processing. San Francisco: Kluwer Academic Publishers.

Digital Library

[3]

S. Bickel, M. Brückner, and T. Scheffer. 2007. Discriminative learning for differing training and test distributions. In Proc. 24th Int. Conf. Machine Learning. 81--88.

Digital Library

[4]

W. Dai, Q. Yang, G. Xue, and Y. Yu. 2007. Boosting for transfer learning. In Proc. 24th Int. Conf. Machine Learning. 193--200.

Digital Library

[5]

W. Dai, Q. Yang, G. Xue, and Y. Yu. 2008. Self-taught clustering. In Proc. 25th Int. Conf. Machine Learning. 200--207.

Digital Library

[6]

J. Davis and P. Domingos. 2008. Deep transfer via second-order markov logic. In Proc. Assoc. for the Advancement of Artificial Intelligence (AAAI’08) Workshop Transfer Learning for Complex Tasks.

[7]

Z. H. Deng, K. S. Choi, F. L. Chung, and S. T. Wang. 2010. Enhanced soft subspace clustering integrating within-cluster and between-cluster information. Pattern Recogn. 43, 3, 767--781.

Digital Library

[8]

Z. H. Deng, K. S. Choi, F. L. Chung, and S. T. Wang. 2011. Scalable TSK fuzzy modeling for very large datasets using minimal-enclosing-ball approximation. IEEE Trans. Fuzzy Syst. 19, 2, 210--226.

Digital Library

[9]

Z. H. Deng, Y. Z. Jiang, F. L. Chung, H. Ishibuchi, and S. T. Wang. 2013a. Knowledge-leverage based fuzzy system and its modeling. IEEE Trans. Fuzzy Syst. 21, 4, 597--609.

Digital Library

[10]

Z. H. Deng, Y. Z. Jiang, K. S. Choi, F. L. Chung, and S. T. Wang. 2013b. Knowledge-leverage-based tsk fuzzy system modeling. IEEE Trans. Neural Netw. Learning Syst. 24, 8, 1200--1212.

[11]

Z. H. Deng, Y. Z. Jiang, L. B. Cao, and S. T. Wang. 2014. Knowledge-leverage based TSK fuzzy system with improved knowledge transfer. In Proceedings of FUZZ-IEEE 2014.

[12]

L. X. Duan, I. W. Tsang, and D. Xu. 2012a. Domain transfer multiple kernel learning. IEEE Trans. Pattern Anal. Mach. Intell. 34, 3, 465--479.

Digital Library

[13]

L. X. Duan, D. Xu, and I. W. Tsang. 2012b. Domain adaptation from multiple sources: A domain-dependent regularization approach. IEEE Trans. Neural Netw. Learning Syst. 23, 3, 504--518.

[14]

R. E. Fan, P. H. Chen, and C. J. Lin. 2005. Working set selection using second order information for training support vector machines. J. Mach. Learning Res. 6, 1889--1918.

Digital Library

[15]

J. Gao, W. Fan, J. Jiang, and J. Han. 2008. Knowledge transfer via multiple model local structure mapping. In Proc. 14th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining. 283--291.

Digital Library

[16]

B. Gong, Y. Shi, F. Sha, and K. Grauman. 2012. Geodesic flow kernel for unsupervised domain adaptation. In Proc. 2012 IEEE Conf. Computer Vision and Pattern Recognition (CVPR’12). IEEE, 2066--2073.

Digital Library

[17]

J. Huang, A. Smola, A. Gretton, K. M. Borgwardt, and B. Schölkopf. 2007. Correcting sample selection bias by unlabeled data. In Proc. 19th Ann. Conf. Neural Information Processing Systems.

Digital Library

[18]

J. S. R. Jang. 1993. ANFIS: Adaptive-network-based fuzzy inference system. IEEE Trans. Systems, Man and Cybernetics 23, 3, 665--685.

[19]

J. S. R. Jang, C. T. Sun, and E. Mizutani. 1997. Neuro-Fuzzy and Soft-Computing. Upper Saddle River, NJ: Prentice-Hall.

Digital Library

[20]

W. H. Jiang and F. L. Chung. 2012. Transfer spectral clustering. In Proc. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD’12).

[21]

L. P. Jing, M. K. Ng, and Z. X. Huang. 2007. An entropy weighting k-means algorithm for subspace clustering of high-dimensional sparse data. IEEE Trans. Knowl. Data Eng. 19, 8, 1026--1041.

Digital Library

[22]

C. F. Juang, S. H. Chiu, and S. J. Shiu. 2007. Fuzzy system learned through fuzzy clustering and support vector machine for human skin color segmentation. IEEE Trans. Syst. Man Cybernetics 37, 6, 1077--1087.

Digital Library

[23]

C. F. Juang and C. D. Hsieh. 2009. TS-fuzzy system-based support vector regression. Fuzzy Sets Syst. 160, 17, 2486--2504.

Digital Library

[24]

N. D. Lawrence and J. C. Platt. 2004. Learning to learn with the informative vector machine. In Proc. 21st Int. Conf. Machine Learning.

Digital Library

[25]

J. Leski. 2005. TSK-fuzzy modeling based on ϵ-insensitive learning. IEEE Trans. Fuzzy Syst. 13, 2, 181--193.

Digital Library

[26]

X. Liao, Y. Xue, and L. Carin. 2005. Logistic regression with an auxiliary data source. In Proc. 21st Int. Conf. Machine Learning. 505--512.

Digital Library

[27]

M. Long, J. Wang, G. Ding, S. J. Pan, and P. S. Yu. 2014. Adaptation regularization: A general framework for transfer learning. IEEE Trans. Knowl. Data Eng. 26, 5, 1076--1089.

Digital Library

[28]

M. Long, J. Wang, J. Sun, and P. S. Yu. 2015. Domain invariant transfer kernel learning. IEEE Trans. Knowl. Data Eng. 27, 6, 1519--1532.

Digital Library

[29]

J. Lu, V. Behbood, P. Hao, H. Zuo, S. Xue, and G. Zhang. 2015. Transfer learning using computational intelligence: A survey. Knowl. Based Syst. 80, 14--23.

Digital Library

[30]

E. Lughofer and S. Kindermann. 2010. SparseFIS: Data-driven learning of fuzzy systems with sparsity constraints. IEEE Trans. Fuzzy Syst. 18, 2, 396--411.

Digital Library

[31]

E. H. Mamdani. 1977. Application of fuzzy logic to approximate reasoning using linguistic synthesis. IEEE Trans. Comput. C-26, 12, 1182--1191.

Digital Library

[32]

J. M. Mendel. 2001. Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions. Upper Saddle River, NJ: Prentice-Hall.

[33]

L. Mihalkova, T. Huynh, and R. J. Mooney. 2007. Mapping and revising Markov logic networks for transfer learning. In Proc. 22nd Assoc. for the Advancement of Artificial Intelligence (AAAI) Conf. Artificial Intelligence. 608--614.

Digital Library

[34]

L. Mihalkova and R. J. Mooney. 2008. Transfer learning by mapping with minimal target data. In Proc. Assoc. for the Advancement of Artificial Intelligence (AAAI’08) Workshop Transfer Learning for Complex Tasks.

[35]

M. Oquab, L. Bottou, I. Laptev, and J. Sivic. 2014. Learning and transferring mid-level image representations using convolutional neural networks. In 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). IEEE, 1717--1724.

Digital Library

[36]

S. J. Pan and Q. Yang. 2010. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 10, 1345--1359.

Digital Library

[37]

S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang. 2011. Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22, 2, 199--210.

Digital Library

[38]

W. Pan and Q. Yang. 2013. Transfer learning in heterogeneous collaborative filtering domains. Artific. Intell. 197, 39--55.

Digital Library

[39]

N. Patricia and B. Caputo. 2014. Learning to learn, from transfer learning to domain adaptation: A unifying perspective. In 2014 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’14). IEEE, 1442--1449.

Digital Library

[40]

B. Quanz and J. Huan. 2009. Large margin transductive transfer learning. In Proceedings of the 18th ACM Conference on Information and Knowledge Management. 1327--1336.

Digital Library

[41]

A. Schwaighofer, V. Tresp, and K. Yu. 2005. Learning Gaussian process kernels via hierarchical Bayes. In Proc. 17th Ann. Conf. Neural Information Processing Systems. 1209--1216.

Digital Library

[42]

L. Shao, F. Zhu, and X. Li. 2015. Transfer learning for visual categorization: A survey. IEEE Trans. Neural Netw. Learning Syst. 26, 5, 1019--1034.

[43]

M. Sugiyama, S. Nakajima, H. Kashima, P. V. Buenau, and M. Kawanabe. 2008. Direct importance estimation with model selection and its application to covariate shift adaptation. In Proc. 20th Ann. Conf. Neural Information Processing Systems.

Digital Library

[44]

T. Takagi and M. Sugeno. 1985. Fuzzy identification of systems and its application to modeling and control. IEEE Trans. Syst. Man Cybernetics 15, 1, 116--132.

[45]

J. W. Tao, K. F. L. Chung, and S. T. Wang. 2012. On minimum distribution discrepancy support vector machine for domain adaptation. Pattern Recogn. 45, 11, 3962--3984.

Digital Library

[46]

T. Tommasi, F. Orabona, and B. Caputo. 2014. Learning categories from few examples with multi model knowledge transfer. IEEE Trans. Pattern Anal. Mach. Intell. 36, 5, 928--941.

[47]

I. W. Tsang, J. T. Kwok, and J. M. Zurada. 2006. Generalized core vector machines. IEEE Trans. Neural Netw. 17, 5, 1126--1140.

Digital Library

[48]

Z. Wang, Y. Song, and C. Zhang. 2008. Transferred dimensionality reduction. In Proc. European Conf. Machine Learning and Knowledge Discovery in Databases (ECML/PKDD’08). 550--565.

[49]

P. Yang, Q. Tan, and Y. Ding. 2008. Bayesian task-level transfer learning for non-linear regression. In Proc. Int. Conf. on Computer Science and Software Engineering. 62--65.

Digital Library

[50]

C. Yang, Z. Deng, K. S. Choi, Y. Jiang, and S. Wang. 2014. Transductive domain adaptive learning for epileptic electroencephalogram recognition. Artif. Intell. Med. 62, 3, 165--177.

[51]

W. Zhang, R. Li, T. Zeng, Q. Sun, S. Kumar, J. Ye, and S. Ji. 2015. Deep model based transfer and multi-task learning for biological image analysis. In Proc. 21th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining. ACM, 1475--1484.

Digital Library

[52]

F. Zhuang, P. Luo, C. Du, Q. He, Z. Shi, and H. Xiong. 2014. Triplex transfer learning: Exploiting both shared and distinct concepts for text classification. IEEE Trans. Cybern. 44, 7, 1191--1203.

[53]

F. Zhuang, X. Cheng, P. Luo, S. J. Pan, and Q. He. 2015. Supervised representation learning: Transfer learning with deep autoencoders. In Proc. 24th International Conference on Artificial Intelligence. AAAI Press, 4119--4125.

Digital Library

Cited By

Lu JMa GZhang G(2024)Fuzzy Machine Learning: A Comprehensive Framework and Systematic ReviewIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.338742932:7(3861-3878)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3387429
Ma GLu JLiu FFang ZZhang G(2024)Domain Adaptation With Interval-Valued Observations: Theory and AlgorithmsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.336746032:5(3107-3120)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3367460
Ma GLu JZhang G(2024)Multisource Domain Adaptation With Interval-Valued Target Data via Fuzzy Neural NetworksIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.336745632:5(3094-3106)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3367456
Show More Cited By

Index Terms

Enhanced Knowledge-Leverage-Based TSK Fuzzy System Modeling for Inductive Transfer Learning
1. Computing methodologies

Recommendations

Bayesian zero-order TSK fuzzy system modeling

Display Omitted The proposed B-ZTSK-FS method can make antecedent and consequent parameters become interpretable and simultaneously tunable.The developed iterative sampling algorithm can realize a fast construction of B-ZTSK-FS for a large-scale ...
TSK-fuzzy modeling based on ε-insensitive learning

In this paper, new learning methods tolerant to imprecision are introduced and applied to fuzzy modeling based on the Takagi-Sugeno-Kang fuzzy system. The fuzzy modeling has an intrinsic inconsistency. It may perform thinking tolerant to imprecision, ...
Center-of-gravity fuzzy systems based on normal fuzzy implications

In this paper, the construction and approximation problem of a single input and single output (SISO) fuzzy system with normal implication and center-of-gravity defuzzifier is discussed. First, the method of a non-singleton fuzzifier for the input ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 8, Issue 1

January 2017

363 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/2973184

Editor:
Yu Zheng
Microsoft Research, China

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2016

Accepted: 01 March 2016

Revised: 01 January 2016

Received: 01 September 2015

Published in TIST Volume 8, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Jiangsu Province Outstanding Youth Fund
Hong Kong Research Grants Council
National Natural Science Foundation of China
Ministry of Education's Program for New Century Excellent Talents
Fundamental Research Funds for the Central Universities

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
363
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lu JMa GZhang G(2024)Fuzzy Machine Learning: A Comprehensive Framework and Systematic ReviewIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.338742932:7(3861-3878)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3387429
Ma GLu JLiu FFang ZZhang G(2024)Domain Adaptation With Interval-Valued Observations: Theory and AlgorithmsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.336746032:5(3107-3120)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3367460
Ma GLu JZhang G(2024)Multisource Domain Adaptation With Interval-Valued Target Data via Fuzzy Neural NetworksIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.336745632:5(3094-3106)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3367456
Ma GLu JLiu FFang ZZhang G(2024)Multiclass Classification With Fuzzy-Feature Observations: Theory and AlgorithmsIEEE Transactions on Cybernetics10.1109/TCYB.2022.318119354:2(1048-1061)Online publication date: Feb-2024
https://doi.org/10.1109/TCYB.2022.3181193
Huong TLan LGiang NBinh NVo BSon L(2023)A novel transfer learning model on complex fuzzy inference systemJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-22258244:3(3733-3750)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/JIFS-222582
Zhu XWang DPedrycz WLi Z(2023)Transfer Learning Realized With Nonlinearly Transformed Input SpaceIEEE Transactions on Emerging Topics in Computing10.1109/TETC.2022.321056811:2(448-460)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TETC.2022.3210568
Si GZi-Zhu FChao XHui W(2023)TSK Fuzzy Model with Integrated Label Smoothing and Feature Smoothing2023 5th International Conference on Industrial Artificial Intelligence (IAI)10.1109/IAI59504.2023.10327598(1-6)Online publication date: 21-Aug-2023
https://doi.org/10.1109/IAI59504.2023.10327598
Ma GLu JZhang G(2023)Interval-Valued Observations-Based Multi-Source Domain Adaptation Using Fuzzy Neural Networks2023 IEEE International Conference on Fuzzy Systems (FUZZ)10.1109/FUZZ52849.2023.10309812(1-6)Online publication date: 13-Aug-2023
https://doi.org/10.1109/FUZZ52849.2023.10309812
Baratta ALongo FMirabelli GPadovano ASolina V(2023)A Digital Twin-Based Approach for Emotion Recognition in Human-Robot CollaborationTowards a Smart, Resilient and Sustainable Industry10.1007/978-3-031-38274-1_14(155-166)Online publication date: 4-Aug-2023
https://doi.org/10.1007/978-3-031-38274-1_14
Sheng HRay BShao JLasantha DDas N(2022)Generalization of solar power yield modeling using knowledge transferExpert Systems with Applications10.1016/j.eswa.2022.116992201(116992)Online publication date: Sep-2022
https://doi.org/10.1016/j.eswa.2022.116992
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents