Abstract
Fiber tractography is a cornerstone of neuroimaging, enabling the detailed mapping of the brain’s white matter pathways through diffusion MRI. This is crucial for understanding brain connectivity and function, making it a valuable tool in neurological applications. Despite its importance, tractography faces challenges due to its complexity and susceptibility to false positives, misrepresenting vital pathways. To address these issues, recent strategies have shifted towards deep learning, utilizing supervised learning, which depends on precise ground truth, or reinforcement learning, which operates without it. In this work, we propose Tract-RLFormer, a network utilizing both supervised and reinforcement learning, in a two-stage policy refinement process that markedly improves the accuracy and generalizability across various data-sets. By employing a tract-specific approach, our network directly delineates the tracts of interest, bypassing the traditional segmentation process. Through rigorous validation on datasets such as TractoInferno, HCP, and ISMRM-2015, our methodology demonstrates a leap forward in tractography, showcasing its ability to accurately map the brain’s white matter tracts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Avants, B.B., et al.: Advanced normalization tools. Insight J 2(365), 1–35 (2009)
Basser, P.J.: Fiber-tractography via diffusion tensor MRI (DT-MRI). In: Proceedings of the 6th Annual Meeting ISMRM, Sydney, Australia, vol. 1226, p. 14 (1998)
Benou, I., Riklin Raviv, T.: Deeptract: A probabilistic deep learning framework for white matter fiber tractography. In: MICCAI: Shenzhen, China, October 13–17, 2019, pp. 626–635. Springer (2019)
Berman, J.I., Chung, S., Mukherjee, P., Hess, C.P., Han, E.T., Henry, R.G.: Probabilistic streamline q-ball tractography using the residual bootstrap. Neuroimage 39(1), 215–222 (2008)
Chen, L., et al.: Decision transformer: reinforcement learning via sequence modeling. NeurIPS 34, 15084–15097 (2021)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Essayed, W.I., Zhang, F., Unadkat, P., Cosgrove, G.R., Golby, A.J., O’Donnell, L.J.: White matter tractography for neurosurgical planning: A topography-based review of the current state of the art. NeuroImage: Clinical 15, 659–672 (2017)
Fillard, P., Poupon, C., Mangin, J.F.: A novel global tractography algorithm based on an adaptive spin glass model. In: MICCAI, pp. 927–934. Springer (2009)
Fujimoto, S., Hoof, H., Meger, D.: Addressing function approximation error in actor-critic methods. In: ICML, pp. 1587–1596. PMLR (2018)
Girard, G., et al.: Towards quantitative connectivity analysis: reducing tractography biases. Neuroimage 98, 266–278 (2014)
Jumper, J., Evans, R., Pritzel, A., Green, T., et al.: Highly accurate protein structure prediction with alphafold. Nature 596(7873), 583–589 (2021)
Maier-Hein, K.H., et al.: The challenge of mapping the human connectome based on diffusion tractography. Nat. Commun. 8(1), 1349 (2017)
Neher, P.F., Côté, M.A., Houde, J.C., Descoteaux, M., Maier-Hein, K.H.: Fiber tractography using machine learning. Neuroimage 158, 417–429 (2017)
Neher, P.F., et al.: A machine learning based approach to fiber tractography using classifier voting. In: MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part I 18, pp. 45–52. Springer (2015)
Poulin, P., et al.: Learn to track: deep learning for tractography. In: MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part I 20, pp. 540–547. Springer (2017)
Poulin, P., et al.: Tractoinferno-a large-scale, open-source, multi-site database for machine learning DMRI tractography. Sci. Data 9(1), 725 (2022)
Rheault, F.: Population average atlas for recobundlesx (2023). https://doi.org/10.5281/zenodo.7950602
Rheault, F., et al.: Bundle-specific tractography with incorporated anatomical and orientational priors. Neuroimage 186, 382–398 (2019)
St-Onge, E., Garyfallidis, E., Collins, D.L.: Fast streamline search: an exact technique for diffusion MRI tractography. Neuroinformatics 20(4), 1093–1104 (2022)
Théberge, A., Desrosiers, C., Boré, A., Descoteaux, M., Jodoin, P.M.: What matters in reinforcement learning for tractography. MIA 93, 103085 (2024)
Théberge, A., et al.: Track-to-learn: a general framework for tractography with deep reinforcement learning. MIA 72, 102093 (2021)
Van Essen, D.C., et al.: The human connectome project: a data acquisition perspective. Neuroimage 62(4), 2222–2231 (2012)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wegmayr, V., Buhmann, J.M.: Entrack: probabilistic spherical regression with entropy regularization for fiber tractography. Int. J. Comput. Vis. 129(3), 656–680 (2021)
Wegmayr, V., Giuliari, G., Holdener, S., Buhmann, J.: Data-driven fiber tractography with neural networks. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp. 1030–1033. IEEE (2018)
Yeh, F.C., et al.: Population-averaged atlas of the macroscale human structural connectome and its network topology. Neuroimage 178, 57–68 (2018)
Zhou, H., Zhang, et al.: Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 11106–11115 (2021)
Acknowledgment
This research was supported by SERB Core Research Grant Project No: CRG/ 2020/005492, IIT Mandi.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Joshi, A. et al. (2025). Tract-RLFormer: A Tract-Specific RL Policy Based Decoder-Only Transformer Network. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15313. Springer, Cham. https://doi.org/10.1007/978-3-031-78201-5_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-78201-5_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78200-8
Online ISBN: 978-3-031-78201-5
eBook Packages: Computer ScienceComputer Science (R0)