research-article

Free access

Massively parallel 3D image reconstruction

Authors:

Putt Sakdhnagool,

Sherman J. Kisner,

Charles A. Bouman,

Samuel P. MidkiffAuthors Info & Claims

SC '17: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Article No.: 3, Pages 1 - 12

https://doi.org/10.1145/3126908.3126911

Published: 12 November 2017 Publication History

Abstract

Computed Tomographic (CT) image reconstruction is an important technique used in a wide range of applications. Among reconstruction methods, Model-Based Iterative Reconstruction (MBIR) is known to produce much higher quality CT images; however, the high computational requirements of MBIR greatly restrict their application. Currently, MBIR speed is primarily limited by irregular data access patterns, the difficulty of effective parallelization, and slow algorithmic convergence.

This paper presents a new algorithm for MBIR, the Non-Uniform Parallel Super-Voxel (NU-PSV) algorithm, that regularizes the data access pattern, enables massive parallelism, and ensures fast convergence. We compare the NU-PSV algorithm with two state-of-the-art implementations on a 69632-core distributed system. Results indicate that the NU-PSV algorithm has an average speedup of 1665 compared to the fastest state-of-the-art implementations.

References

[1]

T. M. Benson, B. D. Man, L. Fu, and J. B. Thibault. 2010. Block-Based Iterative Co-ordinate Descent. In IEEE Nuclear Science Symposuim Medical Imaging Conference. 2856--2859.

[2]

T. Bicer, D. Gürsoy, V. D. Andrade, R. Kettimuthu, W. Scullin, F. D. Carlo, and I. T. Foster. 2017. Trace: A High-Throughput Tomographic Reconstruction Engine for Large-Scale Datasets. Advanced Structural and Chemical Imaging Vol. 3(6) (2017), 1--10.

[3]

C. A. Bouman and K. Sauer. 1996. A Unified Approach to Statistical Tomography Using Coordinate Descent Optimization. IEEE Transactions on Image Processing 5, 3 (March 1996), 480--492.

Digital Library

[4]

J. K. Bradley, A. Kyrola, D. Bickson, and C. Guestrin. 2011. Parallel Coordinate Descent for L₁-Regularized Loss Minimization. In The 28th International Conference on Machine Learning.

Digital Library

[5]

The National Energy Research Scientific Computing Center. 2017. Cori Intel Xeon Phi Nodes. http://www.nersc.gov/users/computational-systems/cori/cori-intel-xeon-phi-nodes/. (2017).

[6]

J. F. Claerbout and F. Muir. 1973. Robust Modeling with Erratic Data. Geophysics 38, 5 (1973), 826--844.

[7]

N. H. Clinthorne, T. S. Pan, P. C. Chiao, W. L. Rogers, and J. A. Stamos. 1993. Preconditioning Methods for Improved Convergence Rates in Iterative Reconstructions. IEEE Transactions on Medical Imaging 12(1) (1993).

[8]

S. Degirmenci, D. G. Politte, C. Bosch, N. Tricha, and J. A. O'Sullivan. 2015. Acceleration of Iterative Image Reconstruction for X-Ray Imaging for Security Applications. In Proceedings of SPIE-IS&T Electronic Imaging, Vol. 9401.

[9]

DHS/ALERT. 2009. Research and Development of Reconstruction Advances in CT-based Object Detection systems. https://myfiles.neu.edu/groups/ALERT/strategic_studies/TO3_FinalReport.pdf. (2009).

[10]

J. Fessler and S. D. Booth. 1999. Conjugate-Gradient Preconditioning Methods for Shift-variant PET Image Reconstruction. IEEE Transactions on Image Processing 8(5) (1999).

Digital Library

[11]

J. A. Fessler, E. P. Ficaro, N. H. Clinthorne, and K. Lange. 1997. Grouped-Coordinate Ascent Algorithms for Penalized-Likelihood Transmission Image Reconstruction. IEEE Transactions on Medical Imaging 16(2) (1997).

[12]

J. A. Fessler and D. Kim. 2011. Axial Block Coordinate Descent (ABCD) Algorithm for X-ray CT Image Reconstruction. In 11th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine.

[13]

J. W. Gibbs, K. A. Mohan, E. B. Gulsoy, A. J. Shahani, X. Xiao, C.A. Bouman, M. De Graef, and P.W. Voorhees. 2015. The Three-Dimensional Morphology of Growing Dendrites. Scientific Reports (2015).

[14]

S. Ha and K. Mueller. 2015. An Algorithm to Compute Independent Sets of Voxels for Parallelization of ICD-based Statistical Iterative Reconstruction. In The 13th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine.

[15]

C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S. Keerthi, and S. Sundararajan. 2008. A Dual Coordinate Descent Method for Large-Scale Linear SVM. In Proceedings of the 25th International Conference on Machine Learning (ICML '08). ACM, New York, NY, USA, 408--415.

Digital Library

[16]

J. Hsieh, B. Nett, Z. Yu, K. Sauer, J.-B. Thibault, and C. A. Bouman. 2013. Recent Advances in CT Image Reconstruction. Current Radiology Reports 1, 1 (2013), 39--51.

[17]

P. Jin, C. A. Bouman, and K. D. Sauer. 2013. A Method for Simultaneous Image Reconstruction and Beam Hardening Correction. In 2013 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC). 1--5.

[18]

P. Jin, E. Haneda, C. A. Bouman, and K. D. Sauer. 2012. A Model-based 3D Multi-slice Helical CT Reconstruction Algorithm for Transportation Security Application. In Second International Conference on Image Formation in X-Ray Computed Tomography.

[19]

S. J. Kisner, E. Haneda, C. A. Bouman, S. Skatter, M. Kourinny, and S. Bedford. 2012. Model-Based CT Reconstruction from Sparse Views. In Second International Conference on Image Formation in X-Ray Computed Tomography. 444--447.

[20]

B. D. Man, S. Basu, J-B. Thibault, J. Hsieh, J. A. Fessler, C. A. Bouman, and K. Sauer. 2005. A Study of Different Minimization Approaches for Iterative Reconstruction in X-ray CT. In IEEE Nuclear Science Symposium, Vol. 5. 2708--2710.

[21]

K. A. Mohan, S. V. Venkatakrishnan, J. W. Gibbs, E. B. Gulsoy, X. Xiao, M. D. Graef, P. W. Voorhees, and C. A. Bouman. 2015. TIMBER: A Method for Time-Space Reconstruction from Interlaced Views. IEEE Transactions on Computational Imaging 1, 2 (June 2015), 96--111.

[22]

Y. Nesterov. 2012. Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems. SIAM Journal on Optimization 22, 2 (2012), 341--362.

[23]

J. Nutini, M. Schmidt, I. H. Laradji, M. Friedlander, and H. Koepke. 2015. Coordinate Descent Converges Faster with the Gauss-southwell Rule Than Random Selection. In The 32nd International Conference on Machine Learning (ICML'15). 1632--1641. http://dl.acm.org/citation.cfm?id=3045118.3045292

Digital Library

[24]

A. Sabne, X. Wang, S. J. Kisner, C. A. Bouman, A. Raghunathan, and S. P. Midkiff. 2017. Model-Based Iterative CT Image Reconstruction on GPUs. In Proceedings of the 22Nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP '17). ACM, New York, NY, USA, 207--220.

Digital Library

[25]

K. Sauer, S. Borman, and C. Bouman. 1995. Parallel Computation of Sequential Pixel Updates in Statistical Tomographic Reconstruction. IEEE Intl. Conf. on Image Processing 3 (October 23-25 1995).

Digital Library

[26]

K. Sauer and C. Bouman. 1993. A Local Update Strategy for Iterative Reconstruction from Projections. IEEE Transactions on Signal Processing 41(2) (1993).

Digital Library

[27]

J. B. Thibault, K. D. Sauer, C. A. Bouman, and J. Hsieh. 2007. A Three-Dimensional Statistical Approach to Improved Image Quality for Multi-Slice Helical CT. Medical Physics 34(11) (2007).

[28]

X. Wang, C. A. Bouman, and S. P. Midkiff. 2017. Equations' Derivations In The Three-Dimensional Super-Voxel Calculations. Purdue University Department of Electrical and Computer Engineering Technical Reports TR-ECE 17-03 (2017).

[29]

X. Wang, K. A. Mohan, S. J. Kisner, C. A. Bouman, and S. P. Midkiff. 2016. Fast Voxel Line Update for Time-Space Image Reconstruction. In The 41st IEEE International Conference on Acoustics, Speech and Signal Processing.

[30]

X. Wang, A. Sabne, S. J. Kisner, A. Raghunathan, C. A. Bouman, and S. P. Midkiff. 2016. High Performance Model Based Image Reconstruction. In 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming.

Digital Library

[31]

S.J. Wright. 2015. Coordinate Descent Algorithms. Math. Program. 151, 1 (June 2015), 3--34.

Digital Library

[32]

L. Wu, P. Babu, and D. P. Palomar. 2017. Cognitive Radar-Based Sequence Design via SINR Maximization. IEEE Transactions on Signal Processing 65(3) (2017).

Digital Library

[33]

Z. Yu, J-B. Thibault, C. A. Bouman, K. D. Sauer, and J. Hsieh. 2011. Fast Model-Based X-Ray CT Reconstruction Using Spatially Nonhomogeneous ICD Optimization. IEEE Transactions on Image Processing 20(1) (2011).

Digital Library

[34]

J. Zheng, S. S. Saquib, K. Sauer, and C. A. Bouman. 2000. Parallelizable Bayesian Tomography Algorithms with Rapid, Guaranteed Convergence. IEEE Transactions on Image Processing 9(10) (2000).

Digital Library

Cited By

Wu DChen PWang XLyngaas IMiyajima TEndo TMatsuoka SWahib M(2024)Real-time High-resolution X-Ray Computed TomographyProceedings of the 38th ACM International Conference on Supercomputing10.1145/3650200.3656634(110-123)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3650200.3656634
Carr ASechrest YMertes KPatterson BWohlberg BHancock LSirica NSandberg RSweeney CHunter JWard WSeaberg MZhu DEsposito VGaltier ESong SBaumann TStadermann MGleason AWeisse-Bernstein N(2024)Morphology of Copper Nanofoams for Radiation Hydrodynamics and Fusion Applications Investigated by 3D PtychotomographyNano Letters10.1021/acs.nanolett.4c0228924:32(9916-9922)Online publication date: 1-Aug-2024
https://doi.org/10.1021/acs.nanolett.4c02289
Hidayetoğlu MBiçer Tde Gonzalo SRen BGürsoy DKettimuthu RFoster IHwu W(2022)MemXCT: Design, Optimization, Scaling, and Reproducibility of X-Ray Tomography ImagingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.312803233:9(2014-2031)Online publication date: 1-Sep-2022
https://doi.org/10.1109/TPDS.2021.3128032
Show More Cited By

Index Terms

Massively parallel 3D image reconstruction
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
      1. Image processing
  2. Parallel computing methodologies
    1. Parallel algorithms
      1. Massively parallel algorithms

Recommendations

Image Reconstruction: Applications in Medical Sciences
Parallel medical image reconstruction: from graphics processing units (GPU) to Grids

We present and compare a variety of parallelization approaches for a real-world case study on modern parallel and distributed computer architectures. Our case study is a production-quality, time-intensive algorithm for medical image reconstruction used ...
A Multistage Parallel Magnetic Resonance Image Reconstruction Method
DICTA '08: Proceedings of the 2008 Digital Image Computing: Techniques and Applications

The length of scan time is a critical issue in Magnetic Resonance Imaging(MRI). Parallel MRI techniques have recently been introduced to shorten the scanning time by using multiple receiver coils to acquire the MR signal. As a result, the imaging ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '17: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

November 2017

801 pages

ISBN:9781450351140

DOI:10.1145/3126908

General Chair:
Bernd Mohr
Jülich Supercomputing Center, Jülich, Germany
,
Program Chair:
Padma Raghavan
Vanderbilt University, Nashville, TN

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Small Business Innovation Research

Conference

SC '17

Sponsor:

SIGHPC

SC '17: The International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 17, 2017

Colorado, Denver

Acceptance Rates

SC '17 Paper Acceptance Rate 61 of 327 submissions, 19%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
865
Total Downloads

Downloads (Last 12 months)112
Downloads (Last 6 weeks)19

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu DChen PWang XLyngaas IMiyajima TEndo TMatsuoka SWahib M(2024)Real-time High-resolution X-Ray Computed TomographyProceedings of the 38th ACM International Conference on Supercomputing10.1145/3650200.3656634(110-123)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3650200.3656634
Carr ASechrest YMertes KPatterson BWohlberg BHancock LSirica NSandberg RSweeney CHunter JWard WSeaberg MZhu DEsposito VGaltier ESong SBaumann TStadermann MGleason AWeisse-Bernstein N(2024)Morphology of Copper Nanofoams for Radiation Hydrodynamics and Fusion Applications Investigated by 3D PtychotomographyNano Letters10.1021/acs.nanolett.4c0228924:32(9916-9922)Online publication date: 1-Aug-2024
https://doi.org/10.1021/acs.nanolett.4c02289
Hidayetoğlu MBiçer Tde Gonzalo SRen BGürsoy DKettimuthu RFoster IHwu W(2022)MemXCT: Design, Optimization, Scaling, and Reproducibility of X-Ray Tomography ImagingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.312803233:9(2014-2031)Online publication date: 1-Sep-2022
https://doi.org/10.1109/TPDS.2021.3128032
Majee SAslan SGursoy DBouman C(2022)CodEx: A Modular Framework for Joint Temporal De-Blurring and Tomographic ReconstructionIEEE Transactions on Computational Imaging10.1109/TCI.2022.31979358(666-678)Online publication date: 2022
https://doi.org/10.1109/TCI.2022.3197935
Venkatakrishnan SMohan KZiabari ABouman C(2022)Algorithm-Driven Advances for Scientific CT Instruments: From model-based to deep learning-based approachesIEEE Signal Processing Magazine10.1109/MSP.2021.312359439:1(32-43)Online publication date: Jan-2022
https://doi.org/10.1109/MSP.2021.3123594
Ye WHuang CHuang JLi JLu YJiang Y(2022)An Integral-equation-oriented Vectorized SpMV Algorithm and its Application on CT Imaging Reconstruction2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS53621.2022.00080(773-783)Online publication date: May-2022
https://doi.org/10.1109/IPDPS53621.2022.00080
Kamble TMahajan SArutwar S(2022)3D Image Reconstruction from Multi-View Images using the Encoder-based Feature Map Generation2022 3rd International Conference on Smart Electronics and Communication (ICOSEC)10.1109/ICOSEC54921.2022.9951956(1497-1503)Online publication date: 20-Oct-2022
https://doi.org/10.1109/ICOSEC54921.2022.9951956
Ali SOtten MDi Z(2022)Accelerating error correction in tomographic reconstructionCommunications Materials10.1038/s43246-022-00267-x3:1Online publication date: 11-Jul-2022
https://doi.org/10.1038/s43246-022-00267-x
Chen PWahib MWang XHirofuchi TOgawa HBiguri ABoardman RBlumensath TMatsuoka Sde Supinski BHall MGamblin T(2021)Scalable FBP decomposition for cone-beam CT reconstructionProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3458817.3476139(1-16)Online publication date: 14-Nov-2021
https://dl.acm.org/doi/10.1145/3458817.3476139
Chen PWahib MWang XTakizawa SHirofuchi TOgawa HMatsuoka SZhou HMoreira JMueller FEtsion Y(2021)Performance portable back-projection algorithms on CPUsProceedings of the 35th ACM International Conference on Supercomputing10.1145/3447818.3460353(316-328)Online publication date: 3-Jun-2021
https://dl.acm.org/doi/10.1145/3447818.3460353
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents