research-article

PromptST: Prompt-Enhanced Spatio-Temporal Multi-Attribute Prediction

Authors:

Zitao LiuAuthors Info & Claims

CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Pages 3195 - 3205

https://doi.org/10.1145/3583780.3615016

Published: 21 October 2023 Publication History

Abstract

In the era of information explosion, spatio-temporal data mining serves as a critical part of urban management. Considering the various fields demanding attention, e.g., traffic state, human activity, and social event, predicting multiple spatio-temporal attributes simultaneously can alleviate regulatory pressure and foster smart city construction. However, current research can not handle the spatio-temporal multi-attribute prediction well due to the complex relationships between diverse attributes. The key challenge lies in how to address the common spatio-temporal patterns while tackling their distinctions. In this paper, we propose an effective solution for spatio-temporal multi-attribute prediction, PromptST. We devise a spatio-temporal transformer and a parameter-sharing training scheme to address the common knowledge among different spatio-temporal attributes. Then, we elaborate a spatio-temporal prompt tuning strategy to fit the specific attributes in a lightweight manner. Through the pretrain and prompt tuning phases, our PromptST is able to enhance the specific spatio-temoral characteristic capture by prompting the backbone model to fit the specific target attribute while maintaining the learned common knowledge. Extensive experiments on real-world datasets verify that our PromptST attains state-of-the-art performance. Furthermore, we also prove PromptST owns good transferability on unseen spatio-temporal attributes, which brings promising application potential in urban computing. The implementation code is available to ease reproducibility.

References

[1]

Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. 2021. Muppet: Massive Multi-task Representations with Pre-Finetuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 5799--5811. https://doi.org/10.18653/v1/2021.emnlp-main.468

[2]

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).

[3]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020a. Language models are few-shot learners. Advances in neural information processing systems, Vol. 33 (2020), 1877--1901.

[4]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020b. Language models are few-shot learners. Advances in neural information processing systems, Vol. 33 (2020), 1877--1901.

[5]

Di Chai, Leye Wang, and Qiang Yang. 2018. Bike flow prediction with multi-graph convolutional networks. In Proceedings of the 26th ACM SIGSPATIAL international conference on advances in geographic information systems. 397--400.

Digital Library

[6]

Changlu Chen, Yanbin Liu, Ling Chen, and Chengqi Zhang. 2022. Bidirectional spatial-temporal adaptive transformer for Urban traffic flow forecasting. IEEE Transactions on Neural Networks and Learning Systems (2022).

[7]

Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making Pre-trained Language Models Better Few-shot Learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, 3816--3830. https://doi.org/10.18653/v1/2021.acl-long.295

[8]

Kan Guo, Yongli Hu, Yanfeng Sun, Sean Qian, Junbin Gao, and Baocai Yin. 2021a. Hierarchical graph convolution network for traffic forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 151--159.

[9]

Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019a. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 922--929.

Digital Library

[10]

Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019b. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 922--929.

Digital Library

[11]

Shengnan Guo, Youfang Lin, Huaiyu Wan, Xiucheng Li, and Gao Cong. 2021b. Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Transactions on Knowledge and Data Engineering, Vol. 34, 11 (2021), 5415--5428.

[12]

Jindong Han, Hao Liu, Hengshu Zhu, Hui Xiong, and Dejing Dou. 2021b. Joint Air Quality and Weather Prediction Based on Multi-Adversarial Spatiotemporal Networks. In Proceedings of the 35th AAAI Conference on Artificial Intelligence.

[13]

Liangzhe Han, Bowen Du, Leilei Sun, Yanjie Fu, Yisheng Lv, and Hui Xiong. 2021a. Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 547--555.

Digital Library

[14]

Xiao Han, Xiangyu Zhao, Liang Zhang, and Wanyu Wang. 2023. Mitigating Action Hysteresis in Traffic Signal Control with Traffic Predictive Reinforcement Learning. 673--684.

[15]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[16]

Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. 2022. Visual Prompt Tuning. In Computer Vision -- ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XXXIII (Tel Aviv, Israel). Springer-Verlag, Berlin, Heidelberg, 709--727. https://doi.org/10.1007/978--3-031--19827--4_41

[17]

Shiyong Lan, Yitong Ma, Weikang Huang, Wenwu Wang, Hongyu Yang, and Pyang Li. 2022. Dstagnn: Dynamic spatial-temporal aware graph neural network for traffic flow forecasting. In International Conference on Machine Learning. PMLR, 11906--11917.

[18]

Can Li, Lei Bai, Wei Liu, Lina Yao, and S Travis Waller. 2020. Knowledge adaption for demand prediction based on multi-task memory neural network. In Proceedings of the 29th ACM international conference on information & knowledge management. 715--724.

Digital Library

[19]

Can Li, Lei Bai, Wei Liu, Lina Yao, and S Travis Waller. 2021. A multi-task memory network with knowledge adaptation for multimodal demand forecasting. Transportation Research Part C: Emerging Technologies, Vol. 131 (2021), 103352.

[20]

Xiang Lisa Li and Percy Liang. 2021. Prefix-Tuning: Optimizing Continuous Prompts for Generation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Vol. abs/2101.00190 (2021).

[21]

Yuxuan Liang, Yutong Xia, Songyu Ke, Yiwei Wang, Qingsong Wen, Junbo Zhang, Yu Zheng, and Roger Zimmermann. 2022. AirFormer: Predicting Nationwide Air Quality in China with Transformers. arXiv preprint arXiv:2211.15979 (2022).

[22]

Hui Liu and Yanfei Li. 2020. Smart cities for emergency management. Nature, Vol. 578, 7796 (2020), 515--516.

[23]

Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. Comput. Surveys, Vol. 55, 9 (2023), 1--35.

Digital Library

[24]

Xiaolei Ma, Zhimin Tao, Yinhai Wang, Haiyang Yu, and Yunpeng Wang. 2015. Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transportation Research Part C: Emerging Technologies, Vol. 54 (2015), 187--197.

[25]

Michael McCloskey and Neal J Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation. Vol. 24. Elsevier, 109--165.

[26]

Fabio Petroni, Tim Rockt"a schel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019. Language Models as Knowledge Bases?. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3--7, 2019. Association for Computational Linguistics, 2463--2473. https://doi.org/10.18653/v1/D19--1250

[27]

Vinay Venkatesh Ramasesh, Aitor Lewkowycz, and Ethan Dyer. 2021. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations.

[28]

Chao Shang and Jie Chen. 2021. Discrete Graph Structure Learning for Forecasting Multiple Time Series. In Proceedings of International Conference on Learning Representations.

[29]

Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. 2022. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1567--1577.

Digital Library

[30]

Zhiqiang Shen, Zechun Liu, Jie Qin, Marios Savvides, and Kwang-Ting Cheng. 2021. Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning. Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 11 (May 2021), 9594--9602. https://doi.org/10.1609/aaai.v35i11.17155

[31]

Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. How to fine-tune bert for text classification?. In China national conference on Chinese computational linguistics. Springer, 194--206.

Digital Library

[32]

Akin Tascikaraoglu. 2018. Evaluation of spatio-temporal forecasting methods in various smart city applications. Renewable and Sustainable Energy Reviews, Vol. 82 (2018), 424--435.

[33]

Yongxue Tian and Li Pan. 2015. Predicting short-term traffic flow by long short-term memory recurrent neural network. In 2015 IEEE international conference on smart city/SocialCom/SustainCom (SmartCity). IEEE, 153--158.

[34]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, Vol. 30 (2017).

[35]

Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In International Conference on Machine Learning. PMLR, 23318--23340.

[36]

Senzhang Wang, Hao Miao, Hao Chen, and Zhiqiu Huang. 2020. Multi-task adversarial spatial-temporal networks for crowd flow prediction. In Proceedings of the 29th ACM international conference on information & knowledge management. 1555--1564.

Digital Library

[37]

Senzhang Wang, Jiaqiang Zhang, Jiyue Li, Hao Miao, and Jiannong Cao. 2021. Traffic Accident Risk Prediction via Multi-View Multi-Task Spatio-Temporal Networks. IEEE Transactions on Knowledge and Data Engineering (2021).

Digital Library

[38]

Yuandong Wang, Hongzhi Yin, Hongxu Chen, Tianyu Wo, Jie Xu, and Kai Zheng. 2019. Origin-destination matrix prediction via graph convolution: a new perspective of passenger demand modeling. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 1227--1235.

Digital Library

[39]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 1907--1913. https://doi.org/10.24963/ijcai.2019/264

[40]

Bin Yan, Yi Jiang, Peize Sun, Dong Wang, Zehuan Yuan, Ping Luo, and Huchuan Lu. 2022. Towards grand unification of object tracking. In Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XXI. Springer, 733--751.

[41]

Haoyang Yan, Xiaolei Ma, and Ziyuan Pu. 2021. Learning dynamic and hierarchical traffic spatiotemporal features with transformer. IEEE Transactions on Intelligent Transportation Systems (2021).

[42]

Junchen Ye, Leilei Sun, Bowen Du, Yanjie Fu, and Hui Xiong. 2021. Coupled layer-wise graph convolution for transportation demand prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 4617--4625.

[43]

Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2017. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875 (2017).

[44]

Chizhan Zhang, Fenghua Zhu, Xiao Wang, Leilei Sun, Haina Tang, and Yisheng Lv. 2020b. Taxi demand prediction using parallel multi-task learning model. IEEE Transactions on Intelligent Transportation Systems (2020).

[45]

Jinlei Zhang, Feng Chen, Yinan Guo, and Xiaohong Li. 2020a. Multi-graph convolutional network for short-term passenger flow forecasting in urban rail transit. IET Intelligent Transport Systems, Vol. 14, 10 (2020), 1210--1217.

[46]

Junbo Zhang, Yu Zheng, and Dekang Qi. 2017. Deep spatio-temporal residual networks for citywide crowd flows prediction. In Thirty-first AAAI conference on artificial intelligence.

Digital Library

[47]

Junbo Zhang, Yu Zheng, Junkai Sun, and Dekang Qi. 2019. Flow prediction in spatio-temporal networks based on multitask deep learning. IEEE Transactions on Knowledge and Data Engineering, Vol. 32, 3 (2019), 468--478.

[48]

Lu Zhang, Zhu Sun, Jie Zhang, Yu Lei, Chen Li, Ziqing Wu, Horst Kloeden, and Felix Klanner. 2021. An interactive multi-task learning framework for next POI recommendation with uncertain check-ins. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence. 3551--3557.

[49]

Zijian Zhang, Xiangyu Zhao, Hao Miao, Chunxu Zhang, Hongwei Zhao, and Junbo Zhang. 2023. AutoSTL: Automated Spatio-Temporal Multi-Task Learning. Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37, 4 (Jun. 2023), 4902--4910. https://doi.org/10.1609/aaai.v37i4.25616

Digital Library

[50]

Shenglin Zhao, Tong Zhao, Haiqin Yang, Michael R Lyu, and Irwin King. 2016b. STELLAR: Spatial-temporal latent ranking for successive point-of-interest recommendation. In Thirtieth AAAI conference on artificial intelligence.

[51]

Xiangyu Zhao, Wenqi Fan, Hui Liu, and Jiliang Tang. 2022. Multi-type Urban Crime Prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 4388--4396.

[52]

Xiangyu Zhao and Jiliang Tang. 2017a. Exploring Transfer Learning for Crime Prediction. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 1158--1159.

[53]

Xiangyu Zhao and Jiliang Tang. 2017b. Modeling Temporal-Spatial Correlations for Crime Prediction. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. ACM, 497--506.

Digital Library

[54]

Xiangyu Zhao and Jiliang Tang. 2018. Crime in Urban Areas:: A Data Mining Perspective. ACM SIGKDD Explorations Newsletter, Vol. 20, 1 (2018), 1--12.

Digital Library

[55]

Xiangyu Zhao, Tong Xu, Yanjie Fu, Enhong Chen, and Hao Guo. 2017. Incorporating Spatio-Temporal Smoothness for Air Quality Inference. In 2017 IEEE International Conference on Data Mining (ICDM). IEEE, 1177--1182.

[56]

Xiangyu Zhao, Tong Xu, Qi Liu, and Hao Guo. 2016a. Exploring the Choice Under Conflict for Social Event Participation. In International Conference on Database Systems for Advanced Applications. Springer, 396--411.

[57]

Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 1234--1241.

[58]

Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. 2015. Forecasting fine-grained air quality based on big data. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 2267--2276.

Digital Library

[59]

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 11106--11115.

Cited By

Li XZhao XWang YLiu YChen CLong CZhang YXing CHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)OpenSiteRec: An Open Dataset for Site RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657875(1483-1493)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657875

Index Terms

PromptST: Prompt-Enhanced Spatio-Temporal Multi-Attribute Prediction
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information systems applications
    1. Spatial-temporal systems
  2. World Wide Web
    1. Web mining
      1. Traffic analysis

Recommendations

A Novel Approach to Identify Spatio-Temporal Crime Pattern in Dhaka City
ICTD '16: Proceedings of the Eighth International Conference on Information and Communication Technologies and Development

Street crime is a prevalent problem in developing countries like Bangladesh. Though this problem has been identified long before, no visible remedy or action can be seen to overcome or combat these street crimes in one of the most populated mega-cities, ...
MetaCitta: Deep Meta-Learning for Spatio-Temporal Prediction Across Cities and Tasks
Advances in Knowledge Discovery and Data Mining
Abstract
Accurate spatio-temporal prediction is essential for capturing city dynamics and planning mobility services. State-of-the-art deep spatio-temporal predictive models depend on rich and representative training data for target regions and tasks. ...
Spatio-Temporal Deep Fusion Graph Convolutional Networks for Crime Prediction
ICMLSC '23: Proceedings of the 2023 7th International Conference on Machine Learning and Soft Computing

Effective crime prediction plays a key role in sustaining the stability of society. In recent years, researchers have proposed a number of prediction methods that extract spatial and temporal features separately and fuse afterward. However, the strict ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

October 2023

5508 pages

ISBN:9798400701245

DOI:10.1145/3583780

General Chairs:
Ingo Frommholz
University of Wolverhampton, UK
,
Frank Hopfgartner
University of Koblenz, Germany
,
Mark Lee
University of Birmingham, UK
,
Michael Oakes
University of Birmingham, UK
,
Program Chairs:
Mounia Lalmas
Spotify, UK
,
Min Zhang
Tsinghua University, China
,
Rodrygo Santos
Federal University of Minas Gerais, Brazil

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

CityU - HKIDS Early Career Research Grant
Natural Science Foundation of Jilin Province
Key Laboratory of Smart Education of Guangdong Higher Education Institutes, Jinan University
Tencent
Kuaishou
Fundamental Research Funds for the Central Universities, JLU
Provincial Science and Technology Innovation Special Fund Project of Jilin Province
APRC - CityU New Research Initiatives
Huawei Technologies
Hong Kong ITC Innovation and Technology Fund Midstream Research Programme for Universities Project
National Key R&D Program of China
Ant Group
SIRG - CityU Strategic Interdisciplinary Research Grant

Conference

CIKM '23

Sponsor:

CIKM '23: The 32nd ACM International Conference on Information and Knowledge Management

October 21 - 25, 2023

Birmingham, United Kingdom

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
398
Total Downloads

Downloads (Last 12 months)398
Downloads (Last 6 weeks)27

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li XZhao XWang YLiu YChen CLong CZhang YXing CHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)OpenSiteRec: An Open Dataset for Site RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657875(1483-1493)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657875

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents