research-article

Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding

Authors:

Valentino Constantinou,

Christopher Laporte,

Tom SoderstromAuthors Info & Claims

KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 387 - 395

https://doi.org/10.1145/3219819.3219845

Published: 19 July 2018 Publication History

Abstract

As spacecraft send back increasing amounts of telemetry data, improved anomaly detection systems are needed to lessen the monitoring burden placed on operations engineers and reduce operational risk. Current spacecraft monitoring systems only target a subset of anomaly types and often require costly expert knowledge to develop and maintain due to challenges involving scale and complexity. We demonstrate the effectiveness of Long Short-Term Memory (LSTMs) networks, a type of Recurrent Neural Network (RNN), in overcoming these issues using expert-labeled telemetry anomaly data from the Soil Moisture Active Passive (SMAP) satellite and the Mars Science Laboratory (MSL) rover, Curiosity. We also propose a complementary unsupervised and nonparametric anomaly thresholding approach developed during a pilot implementation of an anomaly detection system for SMAP, and offer false positive mitigation strategies along with other key improvements and lessons learned during development.

Supplementary Material

MP4 File (hundman_detecting_spacecraft_anomalies.mp4)

Download
357.48 MB

References

[1]

2018. Getting Ready for NISAR-and for Managing Big Data using the Commercial Cloud | Earthdata. https://earthdata.nasa.gov/getting-ready-for-nisar

[2]

Subutai Ahmad, Alexander Lavin, Scott Purdy, and Zuha Agha. 2017. Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262 (2017), 134--147.

[3]

Stephen D. Bay and Mark Schwabacher. 2003. Mining Distance-based Outliers in Near Linear Time with Randomization and a Simple Pruning Rule. In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '03). ACM, New York, NY, USA, 29--38.

Digital Library

[4]

D. Bernard, R. Doyle, E. Riedel, N. Rouquette, J. Wyatt, M. Lowry, and P. Nayak. 1999. Autonomy and software technology on NASA's Deep Space One. IEEE Intelligent Systems 14, 3 (may 1999), 10--15.

Digital Library

[5]

Loic Bontemps, Van Loi Cao, James McDermott, and Nhien-An Le-Khac. 2017. Collective Anomaly Detection based on Long Short Term Memory Recurrent Neural Network. arXiv:arXiv:1703.09752

[6]

Markus Breunig, Hans-Peter Kriegel, Raymond T. Ng, and Järg Sander. 2000. LOF: Identifying Density-Based Local Outliers. In PROCEEDINGS OF THE 2000 ACM SIGMOD INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA. ACM, 93--104.

Digital Library

[7]

Chang, C., Nallo, W., Rastogi, R., Beugless, D., Mickey, F., and Shoop, A. 1992. Satellite diagnostic system: An expert system for intelsat satellite operations. In In Proceedings of the IVth European Aerospace Conference (EAC). 321--327.

[8]

Rich Caruana, Steve Lawrence, and C. Lee Giles. 2001. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In Advances in neural information processing systems. 402--408.

Digital Library

[9]

Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly Detection: A Survey. ACM Comput. Surv. 41, 3, Article 15 (jul 2009), 58 pages.

Digital Library

[10]

Sucheta Chauhan and Lovekesh Vig. 2015. Anomaly detection in ECG time signals via deep long short-term memory networks. In 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA). IEEE.

[11]

F. Ciceri and L. Marradi. 1994. Event diagnosis and recovery in real-time onboard autonomous mission control. In Ada in Europe. Springer Berlin Heidelberg, 288--301.

Digital Library

[12]

RALPH D'AGOSTINO and Egon S. Pearson. 1973. Tests for departure from normality. Empirical results for the distributions of b 2 and b. Biometrika 60, 3 (1973), 613--622.

[13]

Sylvain Fuertes, Gilles Picart, Jean-Yves Tourneret, Lotfi Chaari, André Ferrari, and Cédric Richard. 2016. Improving Spacecraft Health Monitoring with Automatic Anomaly Detection Techniques. In 14th International Conference on Space Operations (SpaceOps 2016). pp--1.

[14]

Ryohei Fujimaki, Takehisa Yairi, and Kazuo Machida. 2005. An Approach to Spacecraft Anomaly Detection Problem Using Kernel Feature Space. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining (KDD '05). ACM, New York, NY, USA, 401--410.

Digital Library

[15]

Yu Gao, Tianshe Yang, Minqiang Xu, and Nan Xing. 2012. An Unsupervised Anomaly Detection Approach for Spacecraft Based on Normal Behavior Clustering. In 2012 Fifth International Conference on Intelligent Computation Technology and Automation. IEEE.

Digital Library

[16]

Markus Goldstein and Seiichi Uchida. 2016. A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data. PLOS ONE 11, 4 (apr 2016), e0152173.

[17]

Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. 2016. Deep learning. Vol. 1. MIT press Cambridge.

Digital Library

[18]

Alex Graves. 2012. Supervised Sequence Labelling with Recurrent Neural Networks. Springer Berlin Heidelberg.

[19]

Alex Graves, Abdel rahman Mohamed, and Geoffrey Hinton. 2013. Speech Recognition with Deep Recurrent Neural Networks. arXiv:arXiv:1303.5778

[20]

K. P. Hand, A. E. Murray, J. B. Garvin, W. B. Brinckerhoff, B. C. Christner, K. S. Edgett, B. L. Ehlmann, C. German, A. G. Hayes, T. M. Hoehler, et al. 2017. Report of the Europa Lander Science Definition Team. Posted February (2017).

[21]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Comput. 9, 8 (nov 1997), 1735--1780.

Digital Library

[22]

J. Stuart Hunter et al. 1986. The exponentially weighted moving average. J. Quality Technol. 18, 4 (1986), 203--210.

[23]

David Iverson. 2008. Data Mining Applications for Space Mission Operations System Health Monitoring. In SpaceOps 2008 Conference. American Institute of Aeronautics and Astronautics.

[24]

David L. Iverson. 2004. Inductive system health monitoring. In In Proceedings of The 2004 International Conference on Artificial Intelligence (IC-AI04), Las Vegas. CSREA Press.

[25]

Hans-Peter Kriegel, Peer Kröger, Erich Schubert, and Arthur Zimek. 2009. LoOP: Local Outlier Probabilities. In Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM '09). ACM, New York, NY, USA, 1649--1652.

Digital Library

[26]

Hans-Peter Kriegel, Peer Kröger, Erich Schubert, and Arthur Zimek. 2009. LoOP: local outlier probabilities. In Proceedings of the 18th ACM conference on Information and knowledge management. ACM, 1649--1652.

Digital Library

[27]

Alexander Lavin and Subutai Ahmad. 2015. Evaluating Real-Time Anomaly Detection Algorithms - The Numenta Anomaly Benchmark. In Machine Learning and Applications (ICMLA), 2015 IEEE 14th International Conference on. IEEE, 38-- 44.

[28]

Ke Li, Yalei Wu, Shimin Song, Yi sun, Jun Wang, and Yang Li. 2016. A novel method for spacecraft electrical fault detection based on FCM clustering and WPSVM classification with PCA feature extraction. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering 231, 1 (aug 2016), 98--108.

[29]

Quan Li, XingShe Zhou, Peng Lin, and Shaomin Li. 2010. Anomaly detection and fault Diagnosis technology of spacecraft based on telemetry-mining. In 2010 3rd International Symposium on Systems and Control in Aeronautics and Astronautics. IEEE.

[30]

Pankaj Malhotra, Vig Lovekesh, Gautam Shroff, and Puneet Argarwal. 2015. Long Short Term Memory Networks for Anomaly Detection in Time Series. In In Proceedings of the European Symposium on Artificial Neural Networks (ESANN), Computational Intelligence and Machine Learning.

[31]

Pankaj Malhotra, Anusha Ramakrishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, and Gautam Shroff. 2016. LSTM-based Encoder-Decoder for Multisensor Anomaly Detection. CoRR abs/1607.00148 (2016).

[32]

Jose Martánez-Heras and Alessandro Donati. 2014. Enhanced Telemetry Monitoring with Novelty Detection. 35 (12 2014), 37--46.

[33]

Anvardh Nanduri and Lance Sherry. 2016. Anomaly detection in aircraft data using Recurrent Neural Networks (RNN). 2016 Integrated Communications Navigation and Surveillance (ICNS) (2016), 5C2-1--5C2-8.

[34]

Naomi Nishigori and Fujitsu Limited. 2001. Fully Automatic and Operator-less Anomaly Detecting Ground Support System For Mars Probe "NOZOMI". In In Proceedings of the 6th International Symposium on Artificial Intelligence and Robotics and Automation in Space (i-SAIRAS).

[35]

Olalekan Ogunmolu, Xuejun Gu, Steve Jiang, and Nicholas Gans. 2016. Nonlinear Systems Identification Using Deep Dynamic Neural Networks. arXiv:arXiv:1610.01439

[36]

M. Rolincikm, Lauriente, M., Koons, H., and D. Gorney. 1992. An expert system for diagnosing environmentally induced spacecraft anomalies. Technical Report. NASA. Lyndon B. Johnson Space Center, Fifth Annual Workshop on Space Operations Applications and Research (SOAR 1991).

[37]

Haím Sak, Andrew Senior, and Françoise Beaufays. 2014. Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition. arXiv:arXiv:1402.1128

[38]

Jürgen Schmidhuber. 2015. Deep learning in neural networks: An overview. Neural Networks 61 (jan 2015), 85--117.

Digital Library

[39]

Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. 1998. Nonlinear Component Analysis As a Kernel Eigenvalue Problem. Neural Comput. 10, 5 (July 1998), 1299--1319.

Digital Library

[40]

R. Sherwood, A. Schlutsmeyer, M. Sue, and E. J. Wyatt. {n. d.}. Lessons from implementation of beacon spacecraft operations on Deep Space One. In 2000 IEEE Aerospace Conference. Proceedings (Cat. No.00TH8484). IEEE.

[41]

Dominique T. Shipmon, Jason M. Gurevitch, Paolo M. Piselli, and Stephen T. Edwards. 2017. Time Series Anomaly Detection; Detection of anomalous drops with limited features and sparse examples in noisy highly periodic data. arXiv:arXiv:1708.03665

[42]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. arXiv:arXiv:1409.3215

Digital Library

[43]

Donald P. Tallo, John Durkin, and Edward J. Petrik. 1992. Intelligent fault isolation and diagnosis for communication satellite systems. Telematics and Informatics 9, 3-4 (jun 1992), 173--190.

Digital Library

[44]

Adrian Taylor, Sylvain Leblanc, and Nathalie Japkowicz. 2016. Anomaly Detection in Automobile Control Network Data with Long Short-Term Memory Networks. In 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA). IEEE.

[45]

Yoshinobu Kawahara Takehisa Yairi. {n. d.}. Telemetry-mining: A Machine Learning Approach to Anomaly Detection and Fault Diagnosis for Space Systems. In 2nd IEEE International Conference on Space Mission Challenges for Information Technology. IEEE.

Digital Library

[46]

Tom Young, Devamanyu Hazarika, Soujanya Poria, and Erik Cambria. 2017. Recent Trends in Deep Learning Based Natural Language Processing. arXiv:arXiv:1708.02709

[47]

Peng Zhou, Zhenyu Qi, Suncong Zheng, Jiaming Xu, Hongyun Bao, and Bo Xu. 2016. Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling. arXiv:arXiv:1611.06639

Cited By

Miao QWang DXia ZXu CZhan JWu C(2025)Exploring spatio-temporal dynamics for enhanced wind turbine condition monitoringMechanical Systems and Signal Processing10.1016/j.ymssp.2024.111841223(111841)Online publication date: Jan-2025
https://doi.org/10.1016/j.ymssp.2024.111841
Gao SWang YZhang ZWang BHe BZio E(2025)Hybrid fault diagnosis method for underwater thrusters based on the common features of multi-source signalsMechanical Systems and Signal Processing10.1016/j.ymssp.2024.111740222(111740)Online publication date: Jan-2025
https://doi.org/10.1016/j.ymssp.2024.111740
Liu JHuang YWu DYang YChen YChen LZhang Y(2024)Multi-Channel Multi-Scale Convolution Attention Variational Autoencoder (MCA-VAE): An Interpretable Anomaly Detection Algorithm Based on Variational AutoencoderSensors10.3390/s2416531624:16(5316)Online publication date: 16-Aug-2024
https://doi.org/10.3390/s24165316
Show More Cited By

Index Terms

Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
1. Applied computing
  1. Operations research
    1. Forecasting
2. Computing methodologies
  1. Machine learning

Recommendations

Detecting Anomalies in Space using Multivariate Convolutional LSTM with Mixtures of Probabilistic PCA
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Detecting an anomaly is not only important for many terrestrial applications on Earth but also for space applications. Especially, satellite missions are highly risky because unexpected hardware and software failures can occur due to sudden or ...
Detecting abnormal behavior in the transportation planning using long short term memories and a contextualized dynamic threshold
UbiComp/ISWC '19 Adjunct: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

Unsupervised anomaly detection in time-series data is crucial for both machine learning research and industrial applications. Over the past few years, the operational efficiencies of logistics agencies have decreased because of a lack of understanding ...
Detecting anomalies in spacecraft telemetry using evolutionary thresholding and LSTMs
GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Detecting anomalies in telemetry data captured on-board satellites is a pivotal step towards their safe operation. The data-driven algorithms for this task are often heavily parameterized, and the incorrect hyperparameters can deteriorate their ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

July 2018

2925 pages

ISBN:9781450355520

DOI:10.1145/3219819

General Chairs:
Yike Guo
Imperial College London
,
Faisal Farooq
IBM

Copyright © 2018 ACM.

© 2018 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the United States Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '18

Sponsor:

KDD '18: The 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 19 - 23, 2018

London, United Kingdom

Acceptance Rates

KDD '18 Paper Acceptance Rate 107 of 983 submissions, 11%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

739
Total Citations
View Citations
5,560
Total Downloads

Downloads (Last 12 months)1,090
Downloads (Last 6 weeks)94

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Miao QWang DXia ZXu CZhan JWu C(2025)Exploring spatio-temporal dynamics for enhanced wind turbine condition monitoringMechanical Systems and Signal Processing10.1016/j.ymssp.2024.111841223(111841)Online publication date: Jan-2025
https://doi.org/10.1016/j.ymssp.2024.111841
Gao SWang YZhang ZWang BHe BZio E(2025)Hybrid fault diagnosis method for underwater thrusters based on the common features of multi-source signalsMechanical Systems and Signal Processing10.1016/j.ymssp.2024.111740222(111740)Online publication date: Jan-2025
https://doi.org/10.1016/j.ymssp.2024.111740
Liu JHuang YWu DYang YChen YChen LZhang Y(2024)Multi-Channel Multi-Scale Convolution Attention Variational Autoencoder (MCA-VAE): An Interpretable Anomaly Detection Algorithm Based on Variational AutoencoderSensors10.3390/s2416531624:16(5316)Online publication date: 16-Aug-2024
https://doi.org/10.3390/s24165316
Rigas STzouveli PKollias S(2024)An End-to-End Deep Learning Framework for Fault Detection in Marine MachinerySensors10.3390/s2416531024:16(5310)Online publication date: 16-Aug-2024
https://doi.org/10.3390/s24165310
Yi SZheng SYang SZhou GCai J(2024)Anomaly Detection for Asynchronous Multivariate Time Series of Nuclear Power Plants Using a Temporal-Spatial TransformerSensors10.3390/s2409284524:9(2845)Online publication date: 29-Apr-2024
https://doi.org/10.3390/s24092845
Zhao MPeng HLi LRen Y(2024)Graph Attention Network and Informer for Multivariate Time Series Anomaly DetectionSensors10.3390/s2405152224:5(1522)Online publication date: 26-Feb-2024
https://doi.org/10.3390/s24051522
Qu XLiu ZWu CHou AYin XChen Z(2024)MFGAN: Multimodal Fusion for Industrial Anomaly Detection Using Attention-Based Autoencoder and Generative Adversarial NetworkSensors10.3390/s2402063724:2(637)Online publication date: 19-Jan-2024
https://doi.org/10.3390/s24020637
Meng FMa LChen YHe WWang ZWang Y(2024)Integrating Sensor Embeddings with Variant Transformer Graph Networks for Enhanced Anomaly Detection in Multi-Source DataMathematics10.3390/math1217261212:17(2612)Online publication date: 23-Aug-2024
https://doi.org/10.3390/math12172612
Wijethunga RNouraei HZych CSamarabandu JSadhu A(2024)Precision Leak Detection in Supermarket Refrigeration Systems Integrating Categorical Gradient Boosting with Advanced ThresholdingEnergies10.3390/en1703073617:3(736)Online publication date: 4-Feb-2024
https://doi.org/10.3390/en17030736
Yang QZhang JZhang JSun CXie SLiu SJi Y(2024)Graph Transformer Network Incorporating Sparse Representation for Multivariate Time Series Anomaly DetectionElectronics10.3390/electronics1311203213:11(2032)Online publication date: 23-May-2024
https://doi.org/10.3390/electronics13112032
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents