Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

A Deep Graph Network with Multiple Similarity for User Clustering in Human–Computer Interaction

Published: 26 September 2023 Publication History
  • Get Citation Alerts
  • Abstract

    User counterparts, such as user attributes in social networks or user interests, are the keys to more natural Human–Computer Interaction (HCI). In addition, users’ attributes and social structures help us understand the complex interactions in HCI. Most previous studies have been based on supervised learning to improve the performance of HCI. However, in the real world, owing to signal malfunctions in user devices, large amounts of abnormal information, unlabeled data, and unsupervised approaches (e.g., the clustering method) based on mining user attributes are particularly crucial. This paper focuses on improving the clustering performance of users’ attributes in HCI and proposes a deep graph embedding network with feature and structure similarity (called DGENFS) to cluster users’ attributes in HCI applications based on feature and structure similarity. The DGENFS model consists of a Feature Graph Autoencoder (FGA) module, a Structure Graph Attention Network (SGAT) module, and a Dual Self-supervision (DSS) module. First, we design an attributed graph clustering method to divide users into clusters by making full use of their attributes. To take full advantage of the information of human feature space, a k-neighbor graph is generated as a feature graph based on the similarity between human features. Then, the FGA and SGAT modules are utilized to extract the representations of human features and topological space, respectively. Next, an attention mechanism is further developed to learn the importance weights of different representations to effectively integrate human features and social structures. Finally, to learn cluster-friendly features, the DSS module unifies and integrates the features learned from the FGA and SGAT modules. DSS explores the high-confidence cluster assignment as a soft label to guide the optimization of the entire network. Extensive experiments are conducted on five real-world data sets on user attribute clustering. The experimental results demonstrate that the proposed DGENFS model achieves the most advanced performance compared with nine competitive baselines.

    References

    [1]
    Jun Wang, Lijun Yin, and Jason Moore. 2007. Using geometric properties of topographic manifold to detect and track eyes for human-computer interaction. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 3, 4 (2007), 1–20.
    [2]
    Zhihan Lv, Alaa Halawani, Shengzhong Feng, Haibo Li, and Shafiq Ur Réhman. 2014. Multimodal hand and foot gesture interaction for handheld devices. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 11, 1s (2014), 1–19.
    [3]
    Alphonse Chapanis. 1965. Man-Machine Engineering.Wadsworth Pub. Co., Inc.
    [4]
    Donald A. Norman and Stephen W. Draper. 1986. User Centered System Design: New Perspectives on Human-Computer Interaction. CUMINCAD, USA.
    [5]
    Christine L. Lisetti. 1998. Affective computing. Pattern Anal. Appl. 1, 1 (1998), 71–73.
    [6]
    Maja Pantic, Anton Nijholt, Alex Pentland, and Thomas S. Huanag. 2008. Human-centred intelligent human? Computer interaction (HCI \(^2\) ): How far are we from attaining it? International Journal of Autonomous and Adaptive Communications Systems 1, 2 (2008), 168–187.
    [7]
    Eleonora Mencarini, Amon Rapp, Lia Tirabeni, and Massimo Zancanaro. 2019. Designing wearable systems for sports: A review of trends and opportunities in human-computer interaction. IEEE Transactions on Human-Machine Systems 49, 4 (2019), 314–325.
    [8]
    Stina Nylander, Jakob Tholander, Florian Mueller, and Joe Marshall. 2014. HCI and sports. (2014), 115–118.
    [9]
    Azin Semsar and Ali Asghar Nazari Shirehjini. 2017. Multimedia-supported virtual experiment for online user–system trust studies. Multimedia Systems 23, 5 (2017), 583–597.
    [10]
    Yun Yang and Jianmin Jiang. 2018. Adaptive bi-weighting toward automatic initialization and model selection for HMM-based hybrid meta-clustering ensembles. IEEE Transactions on Cybernetics 49, 5 (2018), 1657–1668.
    [11]
    Yun Yang and Jianmin Jiang. 2015. Hybrid sampling-based clustering ensemble with global and local constitutions. IEEE Transactions on Neural Networks and Learning Systems 27, 5 (2015), 952–965.
    [12]
    Yun Yang and Jianmin Jiang. 2017. Bi-weighted ensemble via HMM-based approaches for temporal data clustering. Pattern Recognition 76 (2017), 391–403.
    [13]
    Xiaofei He, Deng Cai, Ji-Rong Wen, Wei-Ying Ma, and Hong-Jiang Zhang. 2007. Clustering and searching WWW images using link and page layout analysis. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 3, 2 (2007), 10–es.
    [14]
    Mingxing Duan, Kenli Li, Xiangke Liao, Keqin Li, and Qi Tian. 2019. Features-enhanced multi-attribute estimation with convolutional tensor correlation fusion network. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 15, 3s (2019), 1–23.
    [15]
    Yin Fan, Xiangju Lu, Dian Li, and Yuanliu Liu. 2016. Video-based emotion recognition using CNN-RNN and C3D hybrid networks. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. 445–450.
    [16]
    Arantxa Villanueva, Victoria Ponz, Laura Sesma-Sanchez, Mikel Ariz, Sonia Porta, and Rafael Cabeza. 2013. Hybrid method based on topography for robust detection of iris center and eye corners. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 9, 4 (2013), 1–20.
    [17]
    Shizhe Chen, Qin Jin, Jinming Zhao, and Shuai Wang. 2017. Multimodal multi-task learning for dimensional and continuous emotion recognition. In Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge. 19–26.
    [18]
    André Pimenta, Davide Carneiro, José Neves, and Paulo Novais. 2016. A neural network to classify fatigue from human–computer interaction. Neurocomputing 172 (2016), 413–426.
    [19]
    HongYun Cai, Vincent W. Zheng, and Kevin Chen-Chuan Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering 30, 9 (2018), 1616–1637. DOI:
    [20]
    Santo Fortunato. 2010. Community detection in graphs. Physics Reports 486, 3–5 (2010), 75–174.
    [21]
    Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S. Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems 32, 1 (2020), 4–24.
    [22]
    Fei Tian, Bin Gao, Qing Cui, Enhong Chen, and Tie-Yan Liu. 2014. Learning deep representations for graph clustering. 28, 1 (2014).
    [23]
    Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. (2016), 478–487.
    [24]
    Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. 2017. Improved deep embedded clustering with local structure preservation. In Proceedings of the International Joint Conference on Artificial Intelligence. 1753–1759.
    [25]
    Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1225–1234.
    [26]
    Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. (2015), 1067–1077.
    [27]
    Yu Cao, Meng Fang, and Dacheng Tao. 2019. BAG: Bi-directional attention entity graph convolutional network for multi-hop reasoning question answering. arXiv preprint arXiv:1904.04969 (2019).
    [28]
    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).
    [29]
    Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2020. Graph neural networks: A review of methods and applications. AI Open 1 (2020), 57–81.
    [30]
    Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation learning on graphs with jumping knowledge networks. In Proceedings of the International Conference on Machine Learning. PMLR, 5453–5462.
    [31]
    Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
    [32]
    Thomas N. Kipf and Max Welling. 2016. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016).
    [33]
    Chun Wang, Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Attributed graph clustering: A deep attentional embedding approach. arXiv preprint arXiv:1906.06532 (2019).
    [34]
    Xi Peng, Jiashi Feng, Shijie Xiao, Wei-Yun Yau, Joey Tianyi Zhou, and Songfan Yang. 2018. Structured autoencoders for subspace clustering. IEEE Transactions on Image Processing 27, 10 (2018), 5076–5086.
    [35]
    Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2015. GraRep: Learning graph representations with global structural information. (2015), 891–900.
    [36]
    Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. (2014), 701–710.
    [37]
    Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2016. Deep neural networks for learning graph representations. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.
    [38]
    Zhen Zhang, Hongxia Yang, Jiajun Bu, Sheng Zhou, Pinggang Yu, Jianwei Zhang, Martin Ester, and Can Wang. 2018. ANRL: Attributed network representation learning via deep neural networks. In Proceedings of the International Joint Conference on Artificial Intelligence, Vol. 18. 3155–3161.
    [39]
    Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, and Jing Jiang. 2017. MGAE: Marginalized graph autoencoder for graph clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 889–898.
    [40]
    Xuelong Li, Hongyuan Zhang, and Rui Zhang. 2020. Embedding graph auto-encoder with joint clustering via adjacency sharing. arXiv e-prints (2020), arXiv–2002.
    [41]
    Deyu Bo, Xiao Wang, Chuan Shi, Meiqi Zhu, Emiao Lu, and Peng Cui. 2020. Structural deep clustering network. In Proceedings of the Web Conference 2020. 1400–1410.
    [42]
    Xiaotong Zhang, Han Liu, Xiao-Ming Wu, Xianchao Zhang, and Xinyue Liu. 2021. Spectral embedding network for attributed graph clustering. Neural Networks 142 (2021), 388–396.
    [43]
    Elisabeth Andre. 2013. Exploiting unconscious user signals in multimodal human-computer interaction. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 9, 1s (2013), 1–5.
    [44]
    Shengping Zhang, Huiyu Zhou, Dong Xu, M. Emre Celebi, and Thierry Bouwmans. 2020. Introduction to the Special Issue on Multimodal Machine Learning for Human Behavior Analysis. (2020).
    [45]
    Juan M. Silva, Mauricio Orozco, Jongeun Cha, Abdulmotaleb El Saddik, and Emil M. Petriu. 2013. Human perception of haptic-to-video and haptic-to-audio skew in multimedia applications. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 9, 2 (2013), 1–16.
    [46]
    Sheng Li, Kang Li, and Yun Fu. 2018. Early recognition of 3D human actions. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 14, 1s (2018), 1–21.
    [47]
    Jitao Sang and Changsheng Xu. 2013. Social influence analysis and application on multimedia sharing websites. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 9, 1s (2013), 1–24.
    [48]
    Duc Son Nguyen and Quynh Mai Le. 2020. Hacking user in human-computer interaction design (HCI). In Proceedings of the 2020 3rd International Conference on Information and Computer Technologies (ICICT). IEEE, 230–234.
    [49]
    Wei Shen and Xiaolei Zhou. 2015. Research on the human-computer interaction mode designed for elderly users. In Proceedings of the 2015 4th International Conference on Computer Science and Network Technology (ICCSNT), Vol. 1. 374–377.
    [50]
    Ikuesan R. Adeyemi, Shukor Abd Razak, and Mazleena Salleh. 2016. Individual difference for HCI systems: Examining the probability of thinking style signature in online interaction. In Proceedings of the 2016 4th International Conference on User Science and Engineering (I-User). IEEE, 51–56.
    [51]
    Fereshteh Jadidi Miandashti, Mohammad Izadi, Ali Asghar Nazari Shirehjini, and Shervin Shirmohammadi. 2020. An empirical approach to modeling user-system interaction conflicts in smart homes. IEEE Transactions on Human-Machine Systems 50, 6 (2020), 573–583.
    [52]
    Andrew Ng, Michael Jordan, and Yair Weiss. 2001. On spectral clustering: Analysis and an algorithm. In Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic.
    [53]
    Xiao Wang, Meiqi Zhu, Deyu Bo, Peng Cui, Chuan Shi, and Jian Pei. 2020. AM-GCN: Adaptive multi-channel graph convolutional networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1243–1253.
    [54]
    Van Der Maaten Laurens and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 2605 (2008), 2579–2605.
    [55]
    Giannis Nikolentzos, Polykarpos Meladianos, and Michalis Vazirgiannis. 2017. Matching node embeddings for graph similarity. In Proceedings of the Association for the Advance of Artificial Intelligence.
    [56]
    Matthew B. Hastings. 2006. Community detection as an inference problem. Physical Review E 74, 3 (2006), 035102.

    Cited By

    View all
    • (2024)Structured Deep Graph Clustering Network Based on Consistency ConstraintInternational Journal of Pattern Recognition and Artificial Intelligence10.1142/S0218001424520189Online publication date: 16-Jul-2024
    • (2023)Social-Inspired Multicast Feature Selections with Mobile Edge ComputingGLOBECOM 2023 - 2023 IEEE Global Communications Conference10.1109/GLOBECOM54140.2023.10436726(2287-2292)Online publication date: 4-Dec-2023
    • (2023)PDR-SMOTE: an imbalanced data processing method based on data region partition and K nearest neighborsInternational Journal of Machine Learning and Cybernetics10.1007/s13042-023-01886-714:12(4135-4150)Online publication date: 14-Jun-2023

    Index Terms

    1. A Deep Graph Network with Multiple Similarity for User Clustering in Human–Computer Interaction

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 20, Issue 2
      February 2024
      548 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3613570
      • Editor:
      • Abdulmotaleb El Saddik
      Issue’s Table of Contents

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 26 September 2023
      Online AM: 24 August 2022
      Accepted: 01 May 2022
      Revised: 27 March 2022
      Received: 22 November 2021
      Published in TOMM Volume 20, Issue 2

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Attributed graph clustering
      2. cluster-friendly features
      3. deep graph embedding
      4. self-supervision module

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)216
      • Downloads (Last 6 weeks)17

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Structured Deep Graph Clustering Network Based on Consistency ConstraintInternational Journal of Pattern Recognition and Artificial Intelligence10.1142/S0218001424520189Online publication date: 16-Jul-2024
      • (2023)Social-Inspired Multicast Feature Selections with Mobile Edge ComputingGLOBECOM 2023 - 2023 IEEE Global Communications Conference10.1109/GLOBECOM54140.2023.10436726(2287-2292)Online publication date: 4-Dec-2023
      • (2023)PDR-SMOTE: an imbalanced data processing method based on data region partition and K nearest neighborsInternational Journal of Machine Learning and Cybernetics10.1007/s13042-023-01886-714:12(4135-4150)Online publication date: 14-Jun-2023
      • (2022)An Improved Gray Wolf Optimization Algorithm with a Novel Initialization Method for Community DetectionMathematics10.3390/math1020380510:20(3805)Online publication date: 15-Oct-2022

      View Options

      Get Access

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      Full Text

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media