Abstract
Sequential Monte Carlo (SMC) samplers are a family of powerful Bayesian inference methods that combine sampling and resampling to sample from challenging posterior distributions. This makes SMC widely used in several application domains of statistics and Machine Learning. The aim of this paper is to introduce a new resampling framework, called Conditional Importance Resampling (CIR) that reduces the quantization error arising in the application of traditional resampling schemes. To assess the impact of this approach, we conduct a comparative study between two SMC samplers, differing solely in their resampling schemes: one utilizing systematic resampling and the other employing CIR. The overall improvement is demonstrated by theoretical results and numerical experiments for sampling a forest of Bayesian Decision Trees, focusing on its application in classification and regression tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Arulampalam, M.S., Maskell, S., Gordon, N.J., Clapp, T.: A tutorial on particle filters for online nonlinear/non-gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)
Bacak, A., Hocaoğlu, A.K.: A novel resampling algorithm based on the knapsack problem. Signal Process. 170, 107436 (2020). https://doi.org/10.1016/j.sigpro.2019.107436
Doucet, A., Smith, A., de Freitas, N., Gordon, N.J.: Sequential Monte Carlo Methods in Practice. Information Science and Statistics. Springer, New York (2001)
Drousiotis, E., Varsi, A., Spirakis, P.G., Maskell, S.: A shared memory SMC sampler for decision trees. In: 2023 IEEE 35th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), pp. 209–218 (2023). https://doi.org/10.1109/SBAC-PAD59825.2023.00030
Gordon, N.J., Salmond, D.J., Smith, A.: Novel approach to nonlinear/non-gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process. UK 140(2), 107 (1993)
Green, P.L., Maskell, S.: Parameter estimation from big data using a sequential Monte Carlo sampler. In: Proceedings of ISMA2016 International Conference on Noise and Vibration Engineering and USD2016 International Conference on Uncertainty in Structural Dynamics, pp. 4111–4119 (2016)
Hol, J.D., Schon, T.B., Gustafsson, F.: On resampling algorithms for particle filters. In: 2006 IEEE Nonlinear Statistical Signal Processing Workshop, pp. 79–82 (2006)
Kitagawa, G.: Monte Carlo filter and smoother for non-gaussian nonlinear state space models. J. Comput. Graph. Stat. 5(1), 1–25 (1996)
Kuptametee, C., Aunsri, N.: A review of resampling techniques in particle filtering framework. Measurement 193, 110836 (2022). https://doi.org/10.1016/j.measurement.2022.110836
Li, T., Bolic, M., Djuric, P.M.: Resampling methods for particle filtering: classification, implementation, and strategies. IEEE Signal Process. Mag. 32(3), 70–86 (2015). https://doi.org/10.1109/MSP.2014.2330626
Li, T., Sattar, T.P., Sun, S.: Deterministic resampling: unbiased sampling to avoid sample impoverishment in particle filters. Signal Process. 92(7), 1637–1645 (2012). https://doi.org/10.1016/j.sigpro.2011.12.019
Liu, J.S., Chen, R.: Sequential Monte Carlo methods for dynamic systems. J. Am. Stat. Assoc. 93(443), 1032–1044 (1998)
Lopez, F., Zhang, L., Beaman, J., Mok, A.: Implementation of a particle filter on a GPU for nonlinear estimation in a manufacturing remelting process. In: 2014 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 340–345 (2014). https://doi.org/10.1109/AIM.2014.6878102
Lopez, F., Zhang, L., Mok, A., Beaman, J.: Particle filtering on GPU architectures for manufacturing applications. Comput. Ind. 71, 116–127 (2015). https://doi.org/10.1016/j.compind.2015.03.013
Ma, X., Karkus, P., Hsu, D., Lee, W.S.: Particle filter recurrent neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 5101–5108 (2020)
Mihaylova, L., Carmi, A.Y., Septier, F., Gning, A., Pang, S.K., Godsill, S.: Overview of Bayesian sequential Monte Carlo methods for group and extended object tracking. Digital Signal Process. 25, 1–16 (2014)
Moral, P.D., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. Roy. Stat. Soc. Ser. B (Stat. Methodol.) 68(3), 411–436 (2006)
Murray, L.M., Lee, A., Jacob, P.E.: Parallel resampling in the particle filter. J. Comput. Graph. Stat. 25(3), 789–805 (2016). https://doi.org/10.1080/10618600.2015.1062015
Radojević, M., Meijering, E.: Automated neuron reconstruction from 3D fluorescence microscopy images using sequential Monte Carlo estimation. Neuroinformatics 17(3), 423–442 (2019)
Rosato, C., Varsi, A., Murphy, J., Maskell, S.: An O(log2N) SMC2 algorithm on distributed memory with an approx. optimal l-kernel. In: 2023 IEEE Symposium Sensor Data Fusion and International Conference on Multisensor Fusion and Integration (SDF-MFI), pp. 1–8 (2023). https://doi.org/10.1109/SDF-MFI59545.2023.10361452
Svensson, A., Dahlin, J., Schön, T.B.: 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), pp. 477–480 (2015). https://doi.org/10.1109/CAMSAP.2015.7383840
Varsi, A., Kekempanos, L., Thiyagalingam, J., Maskell, S.: Parallelising particle filters with deterministic runtime on distributed memory systems. In: IET 3rd International Conference on Intelligent Signal Processing (ISP 2017), pp. 1–10 (2017). https://doi.org/10.1049/cp.2017.0357
Varsi, A., Maskell, S., Spirakis, P.G.: An O(log2N) fully-balanced resampling algorithm for particle filters on distributed memory architectures. Algorithms 14(12), 342–362 (2021)
Varsi, A., Taylor, J., Kekempanos, L., Knapp, E.P., Maskell, S.: A fast parallel particle filter for shared memory systems. IEEE Signal Process. Lett. 27, 1570–1574 (2020). https://doi.org/10.1109/LSP.2020.3014035
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Habibi, S., Drousiotis, E., Varsi, A., Maskell, S., Moore, R., Spirakis, P.G. (2025). Conditional Importance Resampling for an Enhanced Sequential Monte Carlo Sampler. In: Festa, P., Ferone, D., Pastore, T., Pisacane, O. (eds) Learning and Intelligent Optimization. LION 2024. Lecture Notes in Computer Science, vol 14990. Springer, Cham. https://doi.org/10.1007/978-3-031-75623-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-75623-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-75622-1
Online ISBN: 978-3-031-75623-8
eBook Packages: Computer ScienceComputer Science (R0)