High Precision ≠ High Cost: Temporal Data Fusion for Multiple Low-Precision Sensors
Authors: 
Jingyu Zhu, Yu Sun, Shaoxu Song, Xiaojie YuanAuthors Info & Claims
Jingyu Zhu, Yu Sun, Shaoxu Song, Xiaojie YuanAuthors Info & ClaimsArticle No.: 143, Pages 1 - 27
Abstract
High-quality data are crucial for practical applications, but obtaining them through high-precision sensors comes at a high cost. To guarantee the trade-off between cost and precision, we may use multiple low-precision sensors to obtain the nearly accurate data fusion results at an affordable cost. The commonly used techniques, such as the Kalman filter and truth discovery methods, typically compute fusion values by combining all the observations according to predictions or sensor reliability. However, low-precision sensors can often cause outliers, and such methods combining all observations are susceptible to interference. To handle this problem, we select a single observation from multiple sensor readings as the fusion result for each timestamp. The selection strategy is guided by the maximum likelihood estimation, to determine the most probable changing trends of fusion results with adjacent timestamps. Our major contributions include (1) the problem formalization and NP-hardness analysis on finding the fusion result with the maximum likelihood w.r.t. local fusion models, (2) exact algorithms based on dynamic programming for tackling the problem, (3) efficient approximation methods with performance guarantees. Experiments on various real datasets and downstream applications demonstrate the superiority and practicality of our work in low-precision sensor data fusion.
Supplemental Material
PPTX File
Presentation slides
- Download
- 12.18 MB
References
[1]
W. A. Abdulhafiz and A. Khamis. Handling data uncertainty and inconsistency using multisensor data fusion. Advances in Artificial Intelligence, 2013:11--11, 2013.
[2]
D. H. Bailey. High-precision floating-point arithmetic in scientific computation. Computing in science & engineering, 7(3):54--61, 2005.
[3]
Y. Bar-Shalom and E. Tse. Tracking in a cluttered environment with probabilistic data association. Automatica, 11(5):451--460, 1975.
[4]
C. M. Bishop and N. M. Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.
[5]
D. R. Brillinger. Time series: data analysis and theory. SIAM, 2001.
[6]
P. Caserman, A. Garcia-Agundez, R. Konrad, S. Göbel, and R. Steinmetz. Real-time body tracking in virtual reality using a vive tracker. Virtual Reality, 23:155--168, 2019.
[7]
F. Castanedo et al. A review of data fusion techniques. The scientific world journal, 2013, 2013.
[8]
C.-Y. Chong. Distributed multitarget multisensor tracking. Multitarget-multisensor tracking: Advanced applications, pages 247--296, 1990.
[9]
M. K. Chung. Gaussian kernel smoothing. arXiv preprint arXiv:2007.09539, 2020.
[10]
T. Cipra and R. Romera. Kalman filter with outliers and missing observations. Test, 6:379--395, 1997.
[11]
E. Clark, S. L. Brunton, and J. N. Kutz. Multi-fidelity sensor selection: Greedy algorithms to place cheap and expensive sensors with cost constraints. IEEE Sensors Journal, 21(1):600--611, 2021.
[12]
D. Crisan and A. Doucet. A survey of convergence results on particle filtering methods for practitioners. IEEE Transactions on signal processing, 50(3):736--746, 2002.
[13]
Y. Deng. Deep learning on mobile devices: a review. In Mobile Multimedia/Image Processing, Security, and Applications 2019, volume 10993, pages 52--66. SPIE, 2019.
[14]
X. L. Dong, L. Berti-Equille, and D. Srivastava. Integrating conflicting data: the role of source dependence. Proceedings of the VLDB Endowment, 2(1):550--561, 2009.
[15]
Y. Dong and Z. Hongyue. Correction of kalman filter in the presence of outlier. In Proceedings of 1994 IEEE International Conference on Industrial Technology-ICIT'94, pages 29--33. IEEE, 1994.
[16]
M. Ester, H.-P. Kriegel, J. Sander, X. Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pages 226--231, 1996.
[17]
C. Fang, S. Song, Y. Mei, Y. Yuan, and J. Wang. On aligning tuples for regression. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 336--346, 2022.
[18]
E. Frank, L. Trigg, G. Holmes, and I. H. Witten. Naive bayes for regression. Machine Learning, 41:5--25, 2000.
[19]
A. Galland, S. Abiteboul, A. Marian, and P. Senellart. Corroborating information from disagreeing views. In Proceedings of the third ACM international conference on Web search and data mining, pages 131--140, 2010.
[20]
L. Golab, H. Karloff, F. Korn, A. Saha, and D. Srivastava. Sequential dependencies. Proceedings of the VLDB Endowment, 2(1):574--585, 2009.
[21]
P. E. Hart, N. J. Nilsson, and B. Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics, 4(2):100--107, 1968.
[22]
A. Jain, E. Y. Chang, and Y.-F. Wang. Adaptive stream resource management using kalman filters. In Proceedings of the 2004 ACM SIGMOD international conference on Management of data, pages 11--22, 2004.
[23]
A. K. Jain and R. C. Dubes. Algorithms for clustering data. Prentice-Hall, Inc., 1988.
[24]
N. Javed and T. Wolf. Automated sensor verification using outlier detection in the internet of things. In 2012 32nd International Conference on Distributed Computing Systems Workshops, pages 291--296. IEEE, 2012.
[25]
S. J. Julier and J. K. Uhlmann. New extension of the kalman filter to nonlinear systems. In Signal processing, sensor fusion, and target recognition VI, volume 3068, pages 182--193. Spie, 1997.
[26]
R. E. Kalman. A new approach to linear filtering and prediction problems. 1960.
[27]
A. Klein. Incorporating quality aspects in sensor data streams. In Proceedings of the ACM first Ph. D. workshop in CIKM, pages 77--84, 2007.
[28]
J. M. Kleinberg. Authoritative sources in a hyperlinked environment. Journal of the ACM (JACM), 46(5):604--632, 1999.
[29]
T. D. Larsen, K. L. Hansen, N. A. Andersen, and O. Ravn. Design of kalman filters for mobile robots; evaluation of the kinematic and odometric approach. In Proceedings of the 1999 IEEE international conference on control applications (Cat. No. 99CH36328), volume 2, pages 1021--1026. IEEE, 1999.
[30]
H. Li, G. Gong, and Q. Du. Prototype of white rabbit network in lhaaso. In Proc. 15th ICALEPCS, pages 1--4, 2015.
[31]
Q. Li, Y. Li, J. Gao, L. Su, B. Zhao, M. Demirbas, W. Fan, and J. Han. A confidence-aware approach for truth discovery on long-tail data. Proceedings of the VLDB Endowment, 8(4):425--436, 2014.
[32]
Q. Li, Y. Li, J. Gao, B. Zhao, W. Fan, and J. Han. Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation. In Proceedings of the 2014 ACM SIGMOD international conference on Management of data, pages 1187--1198, 2014.
[33]
Y. Li, Q. Li, J. Gao, L. Su, B. Zhao, W. Fan, and J. Han. On the discovery of evolving truth. In Proceedings of the 21th acm sigkdd international conference on knowledge discovery and data mining, pages 675--684, 2015.
[34]
C. Liu, P. Shui, G. Wei, and S. Li. Modified unscented kalman filter using modified filter gain and variance scale factor for highly maneuvering target tracking. Journal of Systems Engineering and Electronics, 25(3):380--385, 2014.
[35]
D. Liu, H. Wang, Q. Xia, and C. Jiang. A low-cost method of improving the gnss/sins integrated navigation system using multiple receivers. Electronics, 9(7):1079, 2020.
[36]
J. MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281--297. Oakland, CA, USA, 1967.
[37]
K. Mahmood, S. Niki, Y. Nakahara, X. Lu, I. Luque, and K. Mori. Autonomous real-time navigation for service level agreement in distributed information service system. In Eighth International Symposium on Autonomous Decentralized Systems (ISADS'07), pages 231--238. IEEE, 2007.
[38]
M. Mohri, A. Rostamizadeh, and A. Talwalkar. Foundations of machine learning. MIT press, 2018.
[39]
D. V. Nam and K. Gon-Woo. Robust stereo visual inertial navigation system based on multi-stage outlier removal in dynamic environments. Sensors, 20(10):2922, 2020.
[40]
D. Nemec, J. Andel, V. Simak, and J. Hrbcek. Homogeneous sensor fusion optimization for low-cost inertial sensors. Sensors, 23(14):6431, 2023.
[41]
R. Olfati-Saber. Distributed kalman filtering for sensor networks. In 2007 46th IEEE Conference on Decision and Control, pages 5492--5498. IEEE, 2007.
[42]
J. Pasternack and D. Roth. Knowing what to believe (when you already know something). In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 877--885, 2010.
[43]
Y. Sakurai, M. Yoshikawa, and C. Faloutsos. Ftw: Fast similarity search under the time warping distance. In Proceedings of the Twenty-Fourth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS '05, page 326--337, New York, NY, USA, 2005. Association for Computing Machinery.
[44]
M. Scannapieco and T. Catarci. Data quality under a computer science perspective. Archivi & Computer, 2:1--15, 2002.
[45]
G. Shankaranarayanan and Y. Cai. Supporting data quality management in decision-making. Decision support systems, 42(1):302--317, 2006.
[46]
S. Song, F. Gao, A. Zhang, J. Wang, and P. S. Yu. Stream data cleaning under speed and acceleration constraints. ACM Transactions on Database Systems (TODS), 46(3):1--44, 2021.
[47]
S. Song, A. Zhang, J. Wang, and P. S. Yu. Screen: stream data cleaning under speed constraints. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pages 827--841, 2015.
[48]
R. G. D. Steel, J. H. Torrie, et al. Principles and procedures of statistics. Principles and procedures of statistics., 1960.
[49]
Y. Sun, S. Song, C. Wang, and J. Wang. Swapping repair for misplaced attribute values. In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pages 721--732. IEEE, 2020.
[50]
H. Tang, X. Niu, T. Zhang, J. Fan, and J. Liu. Exploring the accuracy potential of imu preintegration in factor graph optimization. arXiv preprint arXiv:2109.03010, 2021.
[51]
Y.-C. Tseng, M.-S. Pan, and Y.-Y. Tsai. Wireless sensor networks for emergency navigation. Computer, 39(7):55--62, 2006.
[52]
D. Veitch, S. Babu, and A. Pasztor. Robust synchronization of software clocks across the internet. In Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement, pages 219--232, 2004.
[53]
K. Vinther, K. F. Jensen, J. A. Larsen, and R. Wisniewski. Inexpensive cubesat attitude estimation using quaternions and unscented kalman filtering. Automatic Control in Aerospace, 4(1), 2011.
[54]
E. A.Wan and R. Van Der Merwe. The unscented kalman filter for nonlinear estimation. In Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No. 00EX373), pages 153--158. Ieee, 2000.
[55]
R. Y. Wang. A product perspective on total data quality management. Communications of the ACM, 41(2):58--65, 1998.
[56]
R. Y. Wang and D. M. Strong. Beyond accuracy: What data quality means to data consumers. Journal of management information systems, 12(4):5--33, 1996.
[57]
S. Wang, D. Wang, L. Su, L. Kaplan, and T. F. Abdelzaher. Towards cyber-physical systems in social spaces: The data reliability challenge. In 2014 IEEE Real-Time Systems Symposium, pages 74--85, 2014.
[58]
R. Watro, D. Kong, S.-f. Cuti, C. Gardiner, C. Lynn, and P. Kruus. Tinypk: securing sensor networks with public key technology. In Proceedings of the 2nd ACM workshop on Security of ad hoc and sensor networks, pages 59--64, 2004.
[59]
G. Welch, G. Bishop, et al. An introduction to the kalman filter. 1995.
[60]
Y. Xiang, R. Piedrahita, R. P. Dick, M. Hannigan, Q. Lv, and L. Shang. A hybrid sensor system for indoor air quality monitoring. In 2013 IEEE International Conference on Distributed Computing in Sensor Systems, pages 96--104. IEEE, 2013.
[61]
E. Xu, Z. Ding, and S. Dasgupta. Target tracking and mobile sensor navigation in wireless sensor networks. IEEE Transactions on mobile computing, 12(1):177--186, 2011.
[62]
L. Yao, L. Su, Q. Li, Y. Li, F. Ma, J. Gao, and A. Zhang. Online truth discovery on time series data. In Proceedings of the 2018 siam international conference on data mining, pages 162--170. SIAM, 2018.
[63]
C. Yawut and S. Kilaso. A wireless sensor network for weather and disaster alarm systems. In International conference on information and electronics engineering, IPCSIT, volume 6, pages 155--159. Citeseer, 2011.
[64]
B.-K. Yi and C. Faloutsos. Fast time sequence indexing for arbitrary lp norms. 2000.
[65]
X. Yin, J. Han, and P. S. Yu. Truth discovery with multiple conflicting information providers on the web. In Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1048--1052, 2007.
[66]
A. Zhang, S. Song, and J.Wang. Sequential data cleaning: A statistical approach. In Proceedings of the 2016 International Conference on Management of Data, pages 909--924, 2016.
[67]
A. Zhang, S. Song, J. Wang, and P. S. Yu. Time series data cleaning: From anomaly detection to anomaly repairing. Proceedings of the VLDB Endowment, 10(10):1046--1057, 2017.
[68]
B. Zhao and J. Han. A probabilistic model for estimating real-valued truth from conflicting sources. Proc. of QDB, 1817, 2012.
[69]
B. Zhao, B. I. Rubinstein, J. Gemmell, and J. Han. A bayesian approach to discovering truth from conflicting sources for data integration. arXiv preprint arXiv:1203.0058, 2012.
Index Terms
- High Precision ≠ High Cost: Temporal Data Fusion for Multiple Low-Precision Sensors
Recommendations
Construct a robust high-precision detector with multiple low-precision sensors
CCDC'09: Proceedings of the 21st annual international conference on Chinese control and decision conferenceThis paper consider the problem of combining the binary decisions of multiple sensors to produce a better final decision. It assumes that the detection probability and false alarm probability of individual sensors are unknown, but bounded. We prove that ...
Design and Verification of a High-Precision Wireless Strength Testing Node for Aircraft Structure
Wireless sensor network provides a good solution for the state monitoring of large-scale structures. Compared to traditional approaches, it can greatly reduce the amount of devices wiring and the weight of the monitoring system. However, research of ...
On Precision Bound of Distributed Fault-Tolerant Sensor Fusion Algorithms
Sensors have limited precision and accuracy. They extract data from the physical environment, which contains noise. The goal of sensor fusion is to make the final decision robust, minimizing the influence of noise and system errors. One problem that has ...
Comments
Information & Contributors
Information
Published In
June 2024
1953 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 May 2024
Published in PACMMOD Volume 2, Issue 3
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities, Nankai University
- Natural Science Foundation of Tianjin
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 71Total Downloads
- Downloads (Last 12 months)71
- Downloads (Last 6 weeks)16
Reflects downloads up to 30 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in