Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Dynamic trajectory partition optimization method based on historical trajectory data

Published: 17 April 2024 Publication History

Abstract

Partitioning dynamic trajectory data can improve the efficiency and accuracy of trajectory data processing, provide a foundation for trajectory data mining and analysis. However, with the continuous growth of trajectory data scales and the urgent demand for trajectory query efficiency and accuracy, partitioning methods have become crucial. The partitioning method of dynamic trajectory data faces significant challenges in terms of spatiotemporal trajectory locality, partition load balancing, and partition time. To address these challenges, we propose a method based on historical trajectory pre-partitioning, which can store data more effectively in distributed systems. We partition similar historical trajectory data to achieve preliminary partitioning of the data. In addition, we also construct a cost model to ensure that the workload of each partition is close to consistency. Extensive experiments have demonstrated the excellent partitioning efficiency and query efficiency achieved by our design compared to other partitioning methods.

Highlights

We formalize the dynamic trajectory partition problem for trajectory queries.
We propose a cost model for distribute data among nodes.
We transform incremental partitioning into pre-partitioning to avoid data shuffling.
We conduct a comprehensive evaluation of partitioning optimization method.

References

[1]
Petko Bakalov, et al., Time relaxed spatiotemporal trajectory joins, in: ACM SIGSPATIAL International Workshop on Advances in Geographic Information Systems, 2005.
[2]
Frentzos E., Gratsias K., Theodoridis Y., Index-based most similar trajectory search, in: Proceedings - International Conference on Data Engineering, 2007, pp. 816–825,.
[3]
Petko Bakalov, et al., Efficient trajectory joins using symbolic representations, in: International Conference on Mobile Data Management, 2005.
[4]
Lei Chen, et al., Robust and fast similarity search for moving object trajectories, in: ACM SIGMOD Conference, 2005.
[5]
H. Ding, G. Trajcevski, P. Scheuermann, Efficient Similarity Join of Large Sets of Moving Object Trajectories, in: 2008 15th International Symposium on Temporal Representation and Reasoning, Montreal, QC, Canada, 2008, pp. 79–87, https://doi.org/10.1109/TIME.2008.25.
[6]
Wang X., Mueen A., Ding H., Trajcevski G., Scheuermann P., Keogh E., Experimental comparison of representation methods and distance measures for time series data, Data Min. Knowl. Discov. 26 (2) (2013) 275–309,.
[7]
Ding H., Trajcevski G., Scheuermann P., Wang X., Keogh E., Querying and mining of time series data: Experimental comparison of representations and distance measures, Proc. VLDB Endow. 1 (2) (2008),.
[8]
Buchin Kevin, et al., Computing the Fréchet distance between simple polygons, Comput. Geom. 41 (2008) 2–20.
[9]
Driemel Anne, et al., On the discrete Fréchet distance in a graph, 2022, ArXiv abs/2201.02121: n. pag.
[10]
Fushimi Shinya, Kitsuregawa Masaru, Tanaka Hidehiko, An overview of the system software of a parallel relational database machine GRACE, in: Proceedings of the 12th International Conference on Very Large Data Bases (VLDB ’86), Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1986, pp. 209–219.
[11]
DeWitt D.J., Ghandeharizadeh S., Schneider D.A., Bricker A., Hsiao H.-I., Rasmussen R., The Gamma database machine project, IEEE Trans. Knowl. Data Eng. 2 (1) (1990) 44–62,.
[12]
Wen Y., Lu Z., Huang L., Wen Y., Zhang J., Wan C., A storage method of ship trajectory data based on ClickHouse database, in: 2021 6th International Conference on Transportation Information and Safety (ICTIS), Wuhan, China, 2021, pp. 772–779,.
[13]
Zhou X., Li G., Feng J., Liu L., Guo W., Grep: A graph learning based database partitioning system, Proc. ACM Manag. Data 1 (1) (2023) 1–24,.
[14]
Yuan H., Li G., Distributed in-memory trajectory similarity search and join on road network, in: Proceedings - International Conference on Data Engineering, IEEE Computer Society, 2019, pp. 1262–1273,.
[15]
Zhang B.S., Xiang Z.Q., Zhang H., Liu R., Trajectory clustering of segmented field operations logistics process, J. Phys. Conf. Ser. (2021),.
[16]
Alarabi Louai, Summit: a scalable system for massive trajectory data management, SIGSPATIAL Special 10 3 (2018) (2019) 2–3,.
[17]
Xin W., Performance study of large-scale GPS trajectory query method and compression method based on spark platform, in: 2023 IEEE International Conference on Image Processing and Computer Applications (ICIPCA), Changchun, China, 2023, pp. 1–5,.
[18]
Ding J., Fang J., Zhang Z., Zhao P., Xu J., Zhao L., Real-time trajectory similarity processing using longest common subsequence, in: 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), Zhangjiajie, China, 2019, pp. 1398–1405,.
[19]
Zhou Y., Chen Q., Shan B., Jiang F., Pang Y., A distributed storage strategy for trajectory data based on nosql database, in: IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, 2019, pp. 3487–3490,.
[20]
Yue Z., Zhang J., Zhang H., Yang Q., Time-based trajectory data partitioning for efficient range query, in: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Verlag, 2018, pp. 24–35,.
[21]
Y. Liu, Institute of Electrical and Electronics Engineers, and IEEE Circuits and Systems Society, ICNC-FSKD 2017: 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery: Guilin, Guangxi, China, 29-31 July, 2017.
[22]
Kang H., Liu Y., Zhang W., Cloud-based framework for spatio-temporal trajectory data segmentation and query, IEEE Trans. Cloud Comput. 10 (1) (2022) 258–275,.
[23]
Gao Z., Fan Y., Niu K., Wang T., An adaptive initial cluster centers selection algorithm for high-dimensional partition clustering, in: 2017 IEEE 15th Intl Conf on Dependable, Autonomic and Secure Computing, 15th Intl Conf on Pervasive Intelligence and Computing, 3rd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech), Orlando, FL, USA, 2017, pp. 1119–1126,.
[24]
Shang Z., Li G., Bao Z., DITA: Distributed in-memory trajectory analytics, in: Proceedings of the ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, 2018, pp. 725–740,.
[25]
U. Leong Hou, Spaniol Marc, Sakurai Yasushi, Chen Junying, Web and Big Data - 5th International Joint Conference, APWeb-WAIM 2021, Guangzhou, China, August 23-25, 2021, Proceedings, Part I. Lecture Notes in Computer Science 12858, Springer, ISBN 978-3-030-85895-7, 2021.
[26]
2015 IEEE 31st International Conference on Data Engineering: 13-17 April 2015.
[27]
Xie Dong, Li Feifei, Phillips Jeff M., Distributed trajectory similarity search, Proc. VLDB Endow. 10 (11) (2017) 1478–1489,.
[28]
Ailin H., Zhong L., Dechao Z., Cluster-based trajectory overall trend extraction, in: 2017 36th Chinese Control Conference (CCC), Dalian, China, 2017, pp. 5685–5689,.
[29]
Qian W., Zhang Y., Chen Y., Structures of spurious local minima in k-means, IEEE Trans. Inform. Theory 68 (1) (2022) 395–422,.
[30]
Khan S.A., Clustering algorithm on spatiotemporal trajectories, in: 2019 2nd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET, Sukkur, Pakistan, 2019, pp. 1–7,.
[31]
Ganesan Prasanna, Bawa Mayank, Garcia-Molina Hector, Online balancing of range-partitioned data with applications to peer-to-peer systems, in: Proceedings of the Thirtieth International Conference on Very Large Databases - Volume 30 (VLDB ’04), VLDB Endowment, 2004, pp. 444–455.
[32]
Furtado Camille, Lima Alexandre A.B., Pacitti Esther, Valduriez Patrick, Mattoso Marta, Adaptive hybrid partitioning for OLAP query processing in a database cluster, Int. J. High Perform. Comput. Netw. 5 (4) (2008) 251–262,.
[33]
Liroz-Gistau M., Akbarinia R., Pacitti E., Porto F., Valduriez Dynamic P., Valduriez P., Dynamic Workload-Based Partitioning for Large-Scale Databases Workload-Based Partitioning for Large-Scale Databases. DEXA: Database and Expert Systems Ap-Plications Dynamic Workload-Based Partitioning for Large-Scale Databases, 2012, p. 10,.
[34]
Lu Yi, Shanbhag Anil, Jindal Alekh, Madden Samuel, Adaptdb: adaptive partitioning for distributed joins, Proc. VLDB Endow. 10 (5) (2017) 589–600,.
[35]
Shanbhag A., Jindal A., Madden S., Quiane J., Elmore A.J., A robust partitioning scheme for ad-hoc query workloads, in: SoCC 2017 - Proceedings of the 2017 Symposium on Cloud Computing, Association for Computing Machinery, Inc, 2017, pp. 229–241,.
[36]
Y. Gao, Z. Fang, J. Xu, S. Gong, C. Shen, L. Chen, An efficient and distributed framework for real-time trajectory stream clustering, IEEE Trans. Knowl. Data Eng., https://doi.org/10.1109/TKDE.2023.3312319.
[37]
Tian Ruijie, et al., Tinba: Incremental partitioning for efficient trajectory analytics, Adv. Eng. Inform. (2023) n. pag./.
[38]
Li Z., Zhao Z., Mgeohash: Trajectory data index method based on historical data pre-partitioning, in: Proceedings - 2021 7th International Conference on Big Data Computing and Communications, BigCom 2021, Institute of Electrical and Electronics Engineers Inc., 2021, pp. 241–246,.
[39]
Vlachos M., Kollios G., Gunopulos D., Discovering similar multidimensional trajectories, Proc. Int. Conf. Data. Eng. (2002) 673–684,.
[40]
C.D.B. Courtlandt, D.B. Bryan, The National Geographic Society: 100 years of adventure and discovery.
[41]
Wang X., Mueen A., Ding H., Trajcevski G., Scheuermann P., Keogh E., Experimental comparison of representation methods and distance measures for time series data, Data Min. Knowl. Discov. 26 (2) (2013) 275–309,.
[42]
Buchin Kevin, Buchin Maike, Wenk Carola, Computing the Fréchet distance between simple polygons, Comput. Geom. Theory Appl. 41 (1–2) (2008) 2–20,.

Index Terms

  1. Dynamic trajectory partition optimization method based on historical trajectory data
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Applied Soft Computing
      Applied Soft Computing  Volume 151, Issue C
      Jan 2024
      1147 pages

      Publisher

      Elsevier Science Publishers B. V.

      Netherlands

      Publication History

      Published: 17 April 2024

      Author Tags

      1. Pre-partitioning
      2. Dynamic trajectory partitioning
      3. Similarity function

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 0
        Total Downloads
      • Downloads (Last 12 months)0
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 10 Nov 2024

      Other Metrics

      Citations

      View Options

      View options

      Get Access

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media