Abstract
High-dimensional and incomplete (HDI) data usually arise in various complex applications, e.g., bioinformatics and recommender systems, making them commonly heterogeneous and inclusive. Deep neural networks (DNNs)-based approaches have provided state-of-the-art representation learning performance on HDI data. However, most prior studies adopt fixed and exclusive \(L_2\)-norm-oriented loss and regularization terms. Such a single-metric-oriented model yields limited performance on heterogeneous and inclusive HDI data. Motivated by this, we propose a Multi-Metric-Autoencoder (MMA) whose main ideas are two-fold: 1) employing different \(L_p\)-norms to build four variant Autoencoders, each of which resides in a unique metric representation space with different loss and regularization terms, and 2) aggregating these Autoencoders with a tailored, self-adaptive weighting strategy. Theoretical analysis guarantees that our MMA could attain a better representation from a set of dispersed metric spaces. Extensive experiments on four real-world datasets demonstrate that our MMA significantly outperforms seven state-of-the-art models. Our code is available at the link https://github.com/wudi1989/MMA/
This work is supported by the National Natural Science Foundation of China under grant 62176070.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alhayani, B.S., et al.: Visual sensor intelligent module based image transmission in industrial manufacturing for monitoring and manipulation problems. J. Intell. Manuf. 32(2), 597–610 (2021)
Cai, D., Qian, S., Fang, Q., Xu, C.: Heterogeneous hierarchical feature aggregation network for personalized micro-video recommendation. IEEE Trans. Multimedia 24, 805–818 (2021)
Chen, J., Luo, X., Zhou, M.: Hierarchical particle swarm optimization-incorporated latent factor analysis for large-scale incomplete matrices. IEEE Trans. Big Data 8(6), 1524–1536 (2021)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Dong, X., Yu, Z., Cao, W., Shi, Y., Ma, Q.: A survey on ensemble learning. Front. Comp. Sci. 14(2), 241–258 (2020)
Fang, L., Du, B., Wu, C.: Differentially private recommender system with variational autoencoders. Knowl.-Based Syst. 250, 109044 (2022)
Gao, C., et al.: A survey of graph neural networks for recommender systems: Challenges, methods, and directions. ACM Trans. Recommender Syst. (2023). https://doi.org/10.1145/3568022
Gharahighehi, A., Vens, C., Pliakos, K.: Multi-stakeholder news recommendation using hypergraph learning. In: Koprinska, I., et al. (eds.) ECML PKDD 2020. CCIS, vol. 1323, pp. 531–535. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-65965-3_36
Han, S.C., Lim, T., Long, S., Burgstaller, B., Poon, J.: Glocal-k: global and local kernels for recommender systems. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 3063–3067 (2021)
Hu, L., Pan, X., Tang, Z., Luo, X.: A fast fuzzy clustering algorithm for complex networks via a generalized momentum method. IEEE Trans. Fuzzy Syst. 30(9), 3473–3485 (2021)
Hu, L., Yang, S., Luo, X., Yuan, H., Sedraoui, K., Zhou, M.: A distributed framework for large-scale protein-protein interaction data analysis and prediction using mapreduce. IEEE/CAA J. Automatica Sinica 9(1), 160–172 (2021)
Hu, L., Yang, S., Luo, X., Zhou, M.: An algorithm of inductively identifying clusters from attributed graphs. IEEE Trans. Big Data 8(2), 523–534 (2020)
Hu, L., Zhang, J., Pan, X., Luo, X., Yuan, H.: An effective link-based clustering algorithm for detecting overlapping protein complexes in protein-protein interaction networks. IEEE Trans. Netw. Sci. Eng. 8(4), 3275–3289 (2021)
Islek, I., Oguducu, S.G.: A hybrid recommendation system based on bidirectional encoder representations. In: Koprinska, I., et al. (eds.) ECML PKDD 2020. CCIS, vol. 1323, pp. 225–236. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-65965-3_14
Khan, S., Huh, J., Ye, J.C.: Adaptive and compressive beamforming using deep learning for medical ultrasound. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 67(8), 1558–1572 (2020). https://doi.org/10.1109/TUFFC.2020.2977202
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)
Li, P., Wang, Z., Ren, Z., Bing, L., Lam, W.: Neural rating regression with abstractive tips generation for recommendation. In: Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 345–354 (2017)
Li, Y., Sun, H., Yan, W., Cui, Q.: R-CTSVM+: robust capped L1-norm twin support vector machine with privileged information. Inf. Sci. 574, 12–32 (2021)
Li, Z., Li, S., Bamasag, O.O., Alhothali, A., Luo, X.: Diversified regularization enhanced training for effective manipulator calibration. IEEE Transactions on Neural Networks and Learning Systems, pp. 1–13 (2022). https://doi.org/10.1109/TNNLS.2022.3153039
Liu, Z., Luo, X., Wang, Z.: Convergence analysis of single latent factor-dependent, nonnegative, and multiplicative update-based nonnegative latent factor models. IEEE Trans. Neural Netw. Learn. Syst. 32(4), 1737–1749 (2020)
Luo, X., Chen, M., Wu, H., Liu, Z., Yuan, H., Zhou, M.: Adjusting learning depth in nonnegative latent factorization of tensors for accurately modeling temporal patterns in dynamic QoS data. IEEE Trans. Autom. Sci. Eng. 18(4), 2142–2155 (2021)
Luo, X., Wu, H., Li, Z.: NeuLFT: a novel approach to nonlinear canonical polyadic decomposition on high-dimensional incomplete tensors. IEEE Trans. Knowl. Data Eng. 35(6), 6148–6166 (2023)
Luo, X., Zhou, Y., Liu, Z., Hu, L., Zhou, M.: Generalized Nesterov’s acceleration-incorporated, non-negative and adaptive latent factor analysis. IEEE Trans. Serv. Comput. 15(5), 2809–2823 (2021)
Luo, X., Zhou, Y., Liu, Z., Zhou, M.: Fast and accurate non-negative latent factor analysis on high-dimensional and sparse matrices in recommender systems. IEEE Trans. Know. Data Eng. 35, 3897–3911 (2021). https://doi.org/10.1109/TKDE.2021.3125252
Muller, L., Martel, J., Indiveri, G.: Kernelized synaptic weight matrices. In: International Conference on Machine Learning, pp. 3654–3663. PMLR (2018)
Natarajan, S., Vairavasundaram, S., Natarajan, S., Gandomi, A.H.: Resolving data sparsity and cold start problem in collaborative filtering recommender system using linked open data. Expert Syst. Appl. 149, 113248 (2020)
Park, J., Cho, J., Chang, H.J., Choi, J.Y.: Unsupervised hyperbolic representation learning via message passing auto-encoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5516–5526 (2021)
Raza, S., Ding, C.: A regularized model to trade-off between accuracy and diversity in a news recommender system. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 551–560. IEEE (2020)
Saberi-Movahed, F., et al.: Dual regularized unsupervised feature selection based on matrix factorization and minimum redundancy with application in gene selection. Knowl.-Based Syst. 256, 109884 (2022)
Sedhain, S., Menon, A.K., Sanner, S., Xie, L.: AutoRec: autoencoders meet collaborative filtering. In: Proceedings of the 24th International Conference on World Wide Web, pp. 111–112 (2015)
Shang, M., Yuan, Y., Luo, X., Zhou, M.: An \(\alpha \)-\(\beta \)-divergence-generalized recommender for highly accurate predictions of missing user preferences. IEEE Trans. Cybern. 52(8), 8006–8018 (2021)
Shao, B., Li, X., Bian, G.: A survey of research hotspots and frontier trends of recommendation systems from the perspective of knowledge graph. Expert Syst. Appl. 165, 113764 (2021)
Shi, Q., Liu, M., Li, S., Liu, X., Wang, F., Zhang, L.: A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection. IEEE Trans. Geosci. Remote Sens. 60, 1–16 (2022). https://doi.org/10.1109/TGRS.2021.3085870
Shi, X., Kang, Q., An, J., Zhou, M.: Novel L1 regularized extreme learning machine for soft-sensing of an industrial process. IEEE Trans. Ind. Inf. 18(2), 1009–1017 (2021)
Song, Y., Zhu, Z., Li, M., Yang, G., Luo, X.: Non-negative latent factor analysis-incorporated and feature-weighted fuzzy double c-means clustering for incomplete data. IEEE Trans. Fuzzy Syst. 30(10), 4165–4176 (2022)
Tay, Y., Anh Tuan, L., Hui, S.C.: Latent relational metric learning via memory-based attention for collaborative ranking. In: Proceedings of the 2018 World Wide Web Conference, pp. 729–739 (2018)
Wang, H., Hong, Z., Hong, M.: Research on product recommendation based on matrix factorization models fusing user reviews. Appl. Soft Comput. (2022). https://doi.org/10.1016/j.asoc.2022.108971
Wang, S., Cao, J., Yu, P.S.: Deep learning for spatio-temporal data mining: a survey. IEEE Trans. Knowl. Data Eng. 34(8), 3681–3700 (2022). https://doi.org/10.1109/TKDE.2020.3025580
Wang, X., Chen, H., Zhou, Y., Ma, J., Zhu, W.: Disentangled representation learning for recommendation. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 408–424 (2022)
Wu, D., Luo, X.: Robust latent factor analysis for precise representation of high-dimensional and sparse data. IEEE/CAA J. Automatica Sinica 8(4), 796–805 (2021)
Wu, D., Luo, X., He, Y., Zhou, M.: A prediction-sampling-based multilayer-structured latent factor model for accurate representation to high-dimensional and sparse data. IEEE Trans. Neural Netw. Learn. Syst. 1–14 (2022). https://doi.org/10.1109/TNNLS.2022.3200009
Wu, D., Zhang, P., He, Y., Luo, X.: A double-space and double-norm ensembled latent factor model for highly accurate web service QoS prediction. IEEE Trans. Serv. Comput. (2022). https://doi.org/10.1109/TSC.2022.3178543
Wu, H., Luo, X., Zhou, M., Rawa, M.J., Sedraoui, K., Albeshri, A.: A PID-incorporated latent factorization of tensors approach to dynamically weighted directed network analysis. IEEE/CAA J. Automatica Sinica 9(3), 533–546 (2021)
Yuan, Y., He, Q., Luo, X., Shang, M.: A multilayered-and-randomized latent factor model for high-dimensional and sparse matrices. IEEE Trans. Big Data 8(3), 784–794 (2020)
Zhang, M., Chen, Y.: Inductive matrix completion based on graph neural networks. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=ByxxgCEYDS
Zhang, S., Yao, L., Wu, B., Xu, X., Zhang, X., Zhu, L.: Unraveling metric vector spaces with factorization for recommendation. IEEE Trans. Ind. Inf. 16(2), 732–742 (2019)
Zheng, Y., Wang, D.X.: A survey of recommender systems with multi-objective optimization. Neurocomputing 474, 141–153 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liang, C., Wu, D., He, Y., Huang, T., Chen, Z., Luo, X. (2023). MMA: Multi-Metric-Autoencoder for Analyzing High-Dimensional and Incomplete Data. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14173. Springer, Cham. https://doi.org/10.1007/978-3-031-43424-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-43424-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43423-5
Online ISBN: 978-3-031-43424-2
eBook Packages: Computer ScienceComputer Science (R0)
