Article

Free access

The neural hawkes process: a neurally self-modulating multivariate point process

Authors:

Jason EisnerAuthors Info & Claims

NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems

Pages 6757 - 6767

Published: 04 December 2017 Publication History

PDF eReader Publisher Site

Abstract

Many events occur in the world. Some event types are stochastically excited or inhibitedߞin the sense of having their probabilities elevated or decreasedߞby patterns in the sequence of previous events. Discovering such patterns can help us predict which type of event will happen next and when. We model streams of discrete events in continuous time, by constructing a neurally self-modulating multivariate point process in which the intensities of multiple event types evolve according to a novel continuous-time LSTM. This generative model allows past events to influence the future in complex and realistic ways, by conditioning future event intensities on the hidden state of a recurrent neural network that has consumed the stream of past events. Our model has desirable qualitative properties. It achieves competitive likelihood and predictive accuracy on real and synthetic datasets, including under missing-data conditions.

References

[1]

Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. One billion word benchmark for measuring progress in statistical language modeling. Computing Research Repository, arXiv:1312.3005, 2013. URL http://arxiv.org/abs/ 1312.3005.

[2]

Edward Choi, Nan Du, Robert Chen, Le Song, and Jimeng Sun. Constructing disease network and temporal progression model via context-sensitive Hawkes process. In Data Mining (ICDM), 2015 IEEE International Conference on, pages 721-726. IEEE, 2015.

Digital Library

[3]

Nan Du, Mehrdad Farajtabar, Amr Ahmed, Alexander J Smola, and Le Song. Dirichlet-Hawkes processes with applications to clustering continuous-time document streams. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 219-228. ACM, 2015a.

Digital Library

[4]

Nan Du, Yichen Wang, Niao He, Jimeng Sun, and Le Song. Time-sensitive recommendation from recurrent user activities. In Advances in Neural Information Processing Systems (NIPS), pages 3492-3500, 2015b.

[5]

Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1555-1564. ACM, 2016.

Digital Library

[6]

Jalal Etesami, Negar Kiyavash, Kun Zhang, and Kushagra Singhal. Learning network of multivariate Hawkes processes: A time series approach. arXiv preprint arXiv:1603.04319, 2016.

[7]

Manuel Gomez Rodriguez, Jure Leskovec, and Bernhard Schölkopf. Structure and dynamics of information pathways in online media. In Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, pages 23-32. ACM, 2013.

Digital Library

[8]

Alex Graves. Supervised Sequence Labelling with Recurrent Neural Networks. Springer, 2012. URL http://www.cs.toronto.edu/~graves/preprint.pdf.

[9]

Alex Graves, Navdeep Jaitly, and Abdel-rahman Mohamed. Hybrid speech recognition with deep bidirectional LSTM. In Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on, pages 273-278. IEEE, 2013.

[10]

Fangjian Guo, Charles Blundell, Hanna Wallach, and Katherine Heller. The Bayesian echo chamber: Modeling social influence via linguistic accommodation. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, pages 315-323, 2015.

[11]

Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90, 1971.

[12]

Xinran He, Theodoros Rekatsinas, James Foulds, Lise Getoor, and Yan Liu. Hawkestopic: A joint model for network inference and topic modeling from text-based cascades. In Proceedings of the International Conference on Machine Learning (ICML), pages 871-880, 2015.

[13]

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.

Digital Library

[14]

Andrej Karpathy, Justin Johnson, and Li Fei-Fei. Visualizing and understanding recurrent networks. arXiv preprint arXiv:1506.02078, 2015.

[15]

Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR), 2015.

[16]

Young Lee, Kar Wai Lim, and Cheng Soon Ong. Hawkes processes with stochastic excitations. In Proceedings of the International Conference on Machine Learning (ICML), 2016.

[17]

Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data, June 2014.

[18]

Peter A Lewis and Gerald S Shedler. Simulation of nonhomogeneous Poisson processes by thinning. Naval Research Logistics Quarterly, 26(3):403-413, 1979.

[19]

Thomas Josef Liniger. Multivariate Hawkes processes. Diss., Eidgenössische Technische Hochschule ETH Zürich, Nr. 18403, 2009, 2009.

[20]

Michal Lukasik, PK Srijith, Duy Vu, Kalina Bontcheva, Arkaitz Zubiaga, and Trevor Cohn. Hawkes processes for continuous time sequence classification: An application to rumour stance classification in Twitter. In Proceedings of 54th Annual Meeting of the Association for Computational Linguistics, pages 393-398, 2016.

[21]

Tomas Mikolov, Martin Karafiát, Lukás Burget, Jan Cernocký, and Sanjeev Khudanpur. Recurrent neural network based language model. In INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September 26-30, 2010, pages 1045-1048, 2010.

[22]

C. Palm. Intensitätsschwankungen im Fernsprechverkehr. Ericsson technics, no. 44. L. M. Ericcson, 1943. URL https://books.google.com/books?id=5cy2NQAACAAJ.

[23]

Judea Pearl. Causal inference in statistics: An overview. Statistics Surveys, 3:96-146, 2009.

[24]

Martin Sundermeyer, Hermann Ney, and Ralf Schluter. LSTM neural networks for language modeling. Proceedings of INTERSPEECH, 2012.

[25]

Yichen Wang, Bo Xie, Nan Du, and Le Song. Isotonic Hawkes processes. In Proceedings of the International Conference on Machine Learning (ICML), 2016.

[26]

Shuai Xiao, Mehrdad Farajtabar, Xiaojing Ye, Junchi Yan, Xiaokang Yang, Le Song, and Hongyuan Zha. Wasserstein learning of deep generative point process models. In Advances in Neural Information Processing Systems 30, 2017a.

[27]

Shuai Xiao, Junchi Yan, Mehrdad Farajtabar, Le Song, Xiaokang Yang, and Hongyuan Zha. Joint modeling of event sequence and time series with attentional twin recurrent neural networks. arXiv preprint arXiv:1703.08524, 2017b.

[28]

Hongteng Xu, Mehrdad Farajtabar, and Hongyuan Zha. Learning Granger causality for Hawkes processes. In Proceedings of the International Conference on Machine Learning (ICML), 2016.

[29]

Shuang-hong Yang and Hongyuan Zha. Mixture of mutually exciting processes for viral diffusion. In Proceedings of the International Conference on Machine Learning (ICML), pages 1-9, 2013.

[30]

Omar F. Zaidan and Jason Eisner. Modeling annotators: A generative approach to learning from an-notator rationales. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 31-40, 2008.

[31]

Qingyuan Zhao, Murat A Erdogdu, Hera Y He, Anand Rajaraman, and Jure Leskovec. Seismic: A self-exciting point process model for predicting tweet popularity. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1513-1522. ACM, 2015.

Digital Library

[32]

Ke Zhou, Hongyuan Zha, and Le Song. Learning triggering kernels for multi-dimensional Hawkes processes. In Proceedings of the International Conference on Machine Learning (ICML), pages 1301-1309, 2013.

Digital Library

Cited By

Luo DCheng HLi QXu HElkind E(2023)Coupled point process-based sequence modeling for privacy-preserving network alignmentProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/678(6112-6120)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/678
Li RJiang ZGao SYang W(2023)Incorporating Neural Point Process-Based Temporal Feature for Rumor DetectionCombinatorial Optimization and Applications10.1007/978-3-031-49614-1_31(419-430)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1007/978-3-031-49614-1_31
Zhang PIyer RTendulkar AAggarwal GDe ARanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Learning to select exogenous events for marked temporal point processProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3540288(347-361)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3540288
Show More Cited By

The neural hawkes process: a neurally self-modulating multivariate point process
1. Computing methodologies

Recommendations

Event Uncertainty using Ensemble Neural Hawkes Process
CODS-COMAD '23: Proceedings of the 6th Joint International Conference on Data Science & Management of Data (10th ACM IKDD CODS and 28th COMAD)

Various real world applications in science and industry are often recorded over time as asynchronous event sequences. These event sequences comprise of the time of occurrence of events. Different applications including such event sequences are crime ...
Learning Multivariate Hawkes Process via Graph Recurrent Neural Network
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

This paper presents a novel approach for modeling and predicting patterns of events in time-series learning, named graph recurrent temporal point process (GRTPP). Prior research has focused on using deep learning techniques, such as recurrent neural ...
Hawkes Process with Stochastic Triggering Kernel
Advances in Knowledge Discovery and Data Mining
Abstract
The impact from past to future is a vital feature in modelling time series data, which has been described by many point processes, e.g. the Hawkes process. In classical Hawkes process, the triggering kernel is assumed to be a deterministic ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems

December 2017

7104 pages

ISBN:9781510860964

Publisher

Curran Associates Inc.

Red Hook, NY, United States

Publication History

Published: 04 December 2017

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
336
Total Downloads

Downloads (Last 12 months)165
Downloads (Last 6 weeks)31

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Luo DCheng HLi QXu HElkind E(2023)Coupled point process-based sequence modeling for privacy-preserving network alignmentProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/678(6112-6120)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/678
Li RJiang ZGao SYang W(2023)Incorporating Neural Point Process-Based Temporal Feature for Rumor DetectionCombinatorial Optimization and Applications10.1007/978-3-031-49614-1_31(419-430)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1007/978-3-031-49614-1_31
Zhang PIyer RTendulkar AAggarwal GDe ARanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Learning to select exogenous events for marked temporal point processProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3540288(347-361)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3540288
Wang ZJiang RCai ZFan ZLiu XKim KSong XShibasaki RDemartini GZuccon GCulpepper JHuang ZTong H(2021)Spatio-Temporal-Categorical Graph Neural Networks for Fine-Grained Multi-Incident Co-PredictionProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482482(2060-2069)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482482
Zhang YXiong YLi DShan CRen KZhu YDemartini GZuccon GCulpepper JHuang ZTong H(2021)CoPEProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482419(2627-2636)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482419
Li YDing YChen BXin XWang YShi YTang RWang DDemartini GZuccon GCulpepper JHuang ZTong H(2021)Extracting Attentive Social Temporal Excitation for Sequential RecommendationProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482257(998-1007)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482257
Gupta VBedathur SDemartini GZuccon GCulpepper JHuang ZTong H(2021)Region Invariant Normalizing Flows for Mobility TransferProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482169(3053-3057)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482169
Zuo SJiang HLi ZZhao TZha HDaumé HSingh A(2020)Transformer Hawkes processProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3526022(11692-11702)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3526022
Zhang QLipani ALipani AYilmaz EDaumé HSingh A(2020)Self-attentive Hawkes processProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3525975(11183-11193)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3525975
Li SWang LZhang RChang XLiu XXie YQi YSong LDaumé HSingh A(2020)Temporal logic point processesProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3525494(5990-6000)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3525494
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten