Abstract
We present S+t-SNE, an adaptation of the t-SNE algorithm designed to handle infinite data streams. The core idea behind S+t-SNE is to update the t-SNE embedding incrementally as new data arrives, ensuring scalability and adaptability to handle streaming scenarios. By selecting the most important points at each step, the algorithm ensures scalability while keeping informative visualisations. By employing a blind method for drift management, the algorithm adjusts the embedding space, which facilitates the visualisation of evolving data dynamics. Our experimental evaluations demonstrate the effectiveness and efficiency of S+t-SNE, whilst highlighting its ability to capture patterns in a streaming scenario. We hope our approach offers researchers and practitioners a real-time tool for understanding and interpreting high-dimensional data.
P. C. Vieira and J. P. Montrezol—Equal contribution, order defined by coin flip.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(86), 2579–2605 (2008)
Agrahari, S., Singh, A.K.: Concept drift detection in data stream mining : a literature review. J. King Saud Univ. Comput. Inf. Sci. 34(10), 9523–9540 (2022)
Alsakran, J., Chen, Y., Zhao, Y., Yang, J., Luo, D.: StreamIT: dynamic visualization and interactive exploration of text streams. In: 2011 IEEE Pacific Visualization Symposium, pp. 131–138. IEEE (2011)
Basalaj, W.: Incremental multidimensional scaling method for database visualization. In: Visual Data Exploration and Analysis VI, vol. 3643, pp. 149–158. SPIE (1999)
Bengio, Y., Paiement, J., Vincent, P., Delalleau, O., Roux, N., Ouimet, M.: Out-of-sample extensions for LLE, ISOMAP, MDS, eigenmaps, and spectral clustering. Adv. Neural. Inf. Process. Syst. 16, 1–8 (2003)
Jenkins, O.C., Matarić, M.J.: A spatio-temporal extension to Isomap nonlinear dimension reduction. In: Proceedings of the Twenty-first International Conference on Machine Learning, p. 56 (2004)
Joia, P., Coimbra, D., Cuminato, J.A., Paulovich, F.V., Nonato, L.G.: Local affine multidimensional projection. IEEE Trans. Visual Comput. Graph. 17(12), 2563–2571 (2011)
Kouropteva, O., Okun, O., Pietikäinen, M.: Incremental locally linear embedding. Pattern Recogn. 38(10), 1764–1767 (2005)
Law, M.H., Jain, A.K.: Incremental nonlinear dimensionality reduction by manifold learning. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 377–391 (2006)
Law, M.H., Zhang, N., Jain, A.K.: Nonlinear manifold learning for data stream. In: Proceedings of the 2004 SIAM International Conference on Data Mining, pp. 33–44. SIAM (2004)
Maneewongvatana, S., Mount, D.M.: Analysis of approximate nearest neighbor searching with clustered point sets. CoRR cs.CG/9901013 (1999)
McInnes, L., Healy, J., Melville, J.: Umap: Uniform manifold approximation and projection for dimension reduction (2018)
Paulovich, F.V., Nonato, L.G., Minghim, R., Levkowitz, H.: Least square projection: a fast high-precision multidimensional projection technique and its application to document mapping. IEEE Trans. Visual Comput. Graph. 14(3), 564–575 (2008)
Paulovich, F.V., Eler, D.M., Poco, J., Botha, C.P., Minghim, R., Nonato, L.G.: Piece wise Laplacian-based projection for interactive data exploration and organization. In: Computer Graphics Forum, vol. 30, pp. 1091–1100. Wiley Online Library (2011)
Poličar, P.G., Stražar, M., Zupan, B.: openTSNE: a modular python library for t-SNE dimensionality reduction and embedding, August 2019
Rauber, P.E., Falcao, A.X., Telea, A.C., et al.: Visualizing time-dependent data using dynamic t-SNE (2016)
Schuon, S., Durković, M., Diepold, K., Scheuerle, J., Markward, S.: Truly incremental locally linear embedding. In: CoTeSys 1st International Workshop on Cognition for Technical Systems (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
C. Vieira, P., Montrezol, J.P., T. Vieira, J., Gama, J. (2024). S+t-SNE - Bringing Dimensionality Reduction to Data Streams. In: Miliou, I., Piatkowski, N., Papapetrou, P. (eds) Advances in Intelligent Data Analysis XXII. IDA 2024. Lecture Notes in Computer Science, vol 14642. Springer, Cham. https://doi.org/10.1007/978-3-031-58553-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-58553-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-58555-5
Online ISBN: 978-3-031-58553-1
eBook Packages: Computer ScienceComputer Science (R0)