research-article

Knowledge Transfer based on Multiple Manifolds Assumption

Authors:

Yiping KeAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 279 - 287

https://doi.org/10.1145/3357384.3357953

Published: 03 November 2019 Publication History

Abstract

Unsupervised domain adaptation is a popular but challenging problem setting. Existing unsupervised domain adaptation methods are based on the single manifold assumption, i.e., data are sampled from a single low-dimensional manifold, and thus may not well capture the complex characteristic of the real-world data. In this paper, we propose to transfer knowledge across domains under the multiple manifolds assumption that assumes the data are sampled from multiple low-dimensional manifolds. Specifically, we develop a multiple manifolds information transfer framework (MMIT). The proposed MMIT aims to transfer the multiple manifolds information, which is represented by the data manifold neighborhood structure, with the the best adaptation capacity. To do so, we propose to couple the multiple manifolds information transfer with the domain distribution discrepancy minimization in the adaptation procedure. Experimental studies demonstrate that MMIT achieves the promising adaptation performance on various real-world adaptation tasks.

References

[1]

Amir Beck and Marc Teboulle. 2009. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, Vol. 2, 1 (2009), 183--202.

[2]

Karsten M. Borgwardt, Arthur Gr-etton, Malte J. Rasch, Hans-Peter Kriegel, Bernhard Sch-ölkopf, and Alex J Smola. 2006. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics, Vol. 22, 14 (2006), 49--57.

Digital Library

[3]

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et almbox. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine learning, Vol. 3, 1 (2011), 1--122.

[4]

Deng Cai, Xiaofei He, Xuanhui Wang, Hujun Bao, and Jiawei Han. 2009. Locality Preserving Nonnegative Matrix Factorization. In International Joint Conference on Artificial Intelligence. 1010--1015.

[5]

Ehsan Elhamifar and René Vidal. 2009. Sparse subspace clustering. In Computer Vision and Pattern Recognition, IEEE Conference on. 2790--2797.

[6]

Ehsan Elhamifar and Rene Vidal. 2013. Sparse subspace clustering: Algorithm, theory, and applications. Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 35, 11 (2013), 2765--2781.

Digital Library

[7]

Basura Fernando, Amaury Habrard, Marc Sebban, and Tinne Tuytelaars. 2013. Unsupervised visual domain adaptation using subspace alignment. In Computer Vision, IEEE International Conference on. 2960--2967.

[8]

Yaroslav Gani, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francc ois Laviolette, Mario Marchand, and Victor Lempitsky. 2015. Domain-Adversarial Training of Neural Networks. The Journal of Machine Learning Research, Vol. 17, 1 (2015), 2096--2030.

[9]

Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. 2012. Geodesic flow kernel for unsupervised domain adaptation. In Computer Vision and Pattern Recognition, IEEE Conference on. 2066--2073.

[10]

Xiaofei He and Partha Niyogi. 2004. Locality preserving projections. In Advances in Neural Information Processing Systems. 153--160.

[11]

Seung-Jean Kim, Kwangmoo Koh, Lustig Michael, Stephen Boyd, and Dimitry Gorinevsky. 2007. An Interior-Point Method for Large-Scale l1-Regularized Least Squares. IEEE Journal of Selected Topics in Signal Processing, Vol. 1, 4 (2007), 606--617.

[12]

Elizaveta Levina and Peter J Bi-ckel. 2004. Maximum likelihood estimation of intrinsic dimension. In Advances in Neural Information Processing Systems. 777--784.

[13]

Guangcan Liu, Zhouchen Lin, and Yong Yu. 2010. Robust subspace segmentation by low-rank representation. In Proceedings of the 27th international conference on machine learning (ICML-10) . 663--670.

Digital Library

[14]

Mingsheng Long, Yue Cao, Jianmin W-ang, and Michael I Jordan. 2015a. Learning transferable features with deep adaptation networks. In ICML. 97--105.

[15]

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2018. Conditional adversarial domain adaptation. In Advances in Neural Information Processing Systems. 1647--1657.

[16]

Mingsheng Long, Jianmin Wang, Gui-guang Ding, Dou Shen, and Qiang Yang. 2014a. Transfer learning with graph co-regularization. Knowledge and Data Engineering, IEEE Transactions on, Vol. 26, 7 (2014), 1805--1818.

[17]

Mingsheng Long, Jianmin Wang, Gui-guang Ding, Dou Shen, and Qiang Yang. 2015b. Transfer learning with graph co-regularization. Knowledge and Data Engineering, IEEE Transactions on, Vol. 26, 7 (2015), 1805--1818.

[18]

Mingsheng Long, Jianmin Wang, Gui-guang Ding, Jiaguang Sun, and Philip S Yu. 2014b. Transfer Joint Matching for Unsupervised Domain Adaptation. In Computer Vision and Pattern Recognition, IEEE Conference on. 1410--1417.

[19]

Mingsheng Long, Han Zhu, Jianmin Wa-ng, and Michael I. Jordan. 2017. Deep Transfer Learning with Joint Adaptation Networks. In ICML. 2208--2217.

Digital Library

[20]

Zelun Luo, Yuliang Zou, Judy Hoffman, and Li F Fei-Fei. 2017. Label Efficient Learning of Transferable Representations acrosss Domains and Tasks. In NIPS. 164--176.

[21]

Anna Margolis. 2011. A literature review of domain adaptation with unlabeled data. In Report Technique, University of Washington. 1--42.

[22]

Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. 2011. Domain adaptation via transfer component analysis. Neural Networks, IEEE Transactions on, Vol. 22, 2 (2011), 199--210.

Digital Library

[23]

Sam T Roweis and Lawrence K Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science, Vol. 290 (2000), 2323--2326.

[24]

Artem Rozantsev, Mathieu Salzmann, and Pascal Fua. 2019. Beyond sharing weights for deep domain adaptation. IEEE transactions on pattern analysis and machine intelligence, Vol. 41, 4 (2019), 801--814.

Digital Library

[25]

Le Shu, Tianyang Ma, and Longin Jan Latecki. 2014. Locality Preserving Projection for Domain Adaptation with Multi-Objective Learning. In Association for the Advancement of Artificial Intelligence. 2085--2091.

[26]

Baochen Sun, Jiashi Feng, and Kate Sa-enko. 2016. Return of frustratingly easy domain adaptation. In Association for the Advancement of Artificial Intelligence. 2058--2065.

[27]

René Vidal. 2011. Subspace Clustering. IEEE Signal Processing Magazine, Vol. 28, 2 (2011), 52--68.

[28]

Chang Wang and Sridhar Mahadevan. 2011. Heterogeneous domain adaptation using manifold alignment. In International Joint Conference on Artificial Intelligence. 1541--1548.

[29]

Jindong Wang, Wenjie Feng, Yiqiang Chen, Han Yu, Meiyu Huang, and Philip S Yu. 2018. Visual Domain Adaptation with Manifold Embedded Distribution Alignment. In ACM Multimedia Conference (ACM MM). 402--410.

[30]

Pengfei Wei, Yiping Ke, and Chi Keong Goh. 2016. Deep Nonlinear Feature Coding for Unsupervised Domain Adaptation. In International Joint Conference on Artificial Intelligence. 2189--2195.

[31]

Pengfei Wei, Yiping Ke, and Chi Keong Goh. 2017. Domain Specific Feature Transfer for Hybrid Domain Adaptation. In International Conference on Data Mining. 1027--1032.

[32]

Pengfei Wei, Yiping Ke, and Chi Keong Goh. 2018a. Feature Analysis of Marginalized Stacked Denoising Autoenconder for Unsupervised Domain Adaptation. IEEE transactions on neural networks and learning systems (2018), 1--14.

[33]

Pengfei Wei, Yiping Ke, and Chi Keong Goh. 2018b. A General Domain Specific Feature Transfer Framework for Hybrid Domain Adaptation. IEEE Transactions on Knowledge and Data Engineering (2018), 1440 -- 1451.

[34]

Jing Zhang, Wanqing Li, and Philip Ogunbona. 2017. Joint geometrical and statistical alignment for visual domain adaptation. In CVPR . 692--701.

[35]

Xu Zhang, Felix Xinnan Yu, Shih-Fu Chang, and Shengjin Wang. 2015. Deep Transfer Network: Unsupervised Domain Adaptation. arXiv:1503.00591 (2015).

Cited By

Wei PKe YQu XLeong T(2022)Subdomain Adaptation With Manifolds Discrepancy AlignmentIEEE Transactions on Cybernetics10.1109/TCYB.2021.307124452:11(11698-11708)Online publication date: Nov-2022
https://doi.org/10.1109/TCYB.2021.3071244
Yang SYu KCao FWang HWu X(2022)Dual-Representation-Based Autoencoder for Domain AdaptationIEEE Transactions on Cybernetics10.1109/TCYB.2020.304076352:8(7464-7477)Online publication date: Aug-2022
https://doi.org/10.1109/TCYB.2020.3040763
Wei PLeong T(2021)Randomized Transferable Machine2020 25th International Conference on Pattern Recognition (ICPR)10.1109/ICPR48806.2021.9413321(8711-8718)Online publication date: 10-Jan-2021
https://doi.org/10.1109/ICPR48806.2021.9413321
Show More Cited By

Index Terms

Knowledge Transfer based on Multiple Manifolds Assumption
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Multi-task learning
        Transfer learning

Recommendations

Learning representations from multiple manifolds

The problem we address in this paper is how to learn joint representation from data lying on multiple manifolds. We are given multiple data sets, and there is an underlying common manifold among the different data sets. Each data set is considered to be ...
Facial expression recognition on multiple manifolds

Manifold learning has been successfully applied to facial expression recognition by modeling different expressions as a smooth manifold embedded in a high dimensional space. However, the assumption of single manifold is still arguable and therefore does ...
Geometrically local embedding in manifolds for dimension reduction

In this paper, geometrically local embedding (GLE) is presented to discover the intrinsic structure of manifolds as a method in nonlinear dimension reduction. GLE is able to reveal the inner features of the input data in the lower dimension space while ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
255
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wei PKe YQu XLeong T(2022)Subdomain Adaptation With Manifolds Discrepancy AlignmentIEEE Transactions on Cybernetics10.1109/TCYB.2021.307124452:11(11698-11708)Online publication date: Nov-2022
https://doi.org/10.1109/TCYB.2021.3071244
Yang SYu KCao FWang HWu X(2022)Dual-Representation-Based Autoencoder for Domain AdaptationIEEE Transactions on Cybernetics10.1109/TCYB.2020.304076352:8(7464-7477)Online publication date: Aug-2022
https://doi.org/10.1109/TCYB.2020.3040763
Wei PLeong T(2021)Randomized Transferable Machine2020 25th International Conference on Pattern Recognition (ICPR)10.1109/ICPR48806.2021.9413321(8711-8718)Online publication date: 10-Jan-2021
https://doi.org/10.1109/ICPR48806.2021.9413321
Abdi LHashemi S(2021)Unsupervised Domain Adaptation Based on Correlation MaximizationIEEE Access10.1109/ACCESS.2021.31115869(127054-127067)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3111586
Wei PKe YXu ZLeong Td'Aquin MDietze SHauff CCurry ECudre Mauroux P(2020)Succinct Adaptive Manifold TransferProceedings of the 29th ACM International Conference on Information & Knowledge Management10.1145/3340531.3411921(1615-1624)Online publication date: 19-Oct-2020
https://dl.acm.org/doi/10.1145/3340531.3411921

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten