Article

Scalable algorithms for molecular dynamics simulations on commodity clusters

Authors:

Kevin J. Bowers,

Michael P. Eastwood,

Brent A. Gregersen,

John L. Klepeis,

Istvan Kolossvary,

Mark A. Moraes,

Federico D. Sacerdoti,

John K. Salmon,

David E. ShawAuthors Info & Claims

SC '06: Proceedings of the 2006 ACM/IEEE conference on Supercomputing

Pages 84 - es

https://doi.org/10.1145/1188455.1188544

Published: 11 November 2006 Publication History

Abstract

Although molecular dynamics (MD) simulations of biomolecular systems often run for days to months, many events of great scientific interest and pharmaceutical relevance occur on long time scales that remain beyond reach. We present several new algorithms and implementation techniques that significantly accelerate parallel MD simulations compared with current state-of-the-art codes. These include a novel parallel decomposition method and message-passing techniques that reduce communication requirements, as well as novel communication primitives that further reduce communication time. We have also developed numerical techniques that maintain high accuracy while using single precision computation in order to exploit processor-level vector instructions. These methods are embodied in a newly developed MD code called Desmond that achieves unprecedented simulation throughput and parallel scalability on commodity clusters. Our results suggest that Desmond's parallel performance substantially surpasses that of any previously described code. For example, on a standard benchmark, Desmond's performance on a conventional Opteron cluster with 2K processors slightly exceeded the reported performance of IBM's Blue Gene/L machine with 32K processors running its Blue Matter MD code.

References

[1]

G. Almasi, C. Archer, J. G. Castanos, et al., Design and Implementation of Message-Passing Services for the Blue Gene/L Supercomputer, IBM J. Res. & Dev., 49(2-3): 393--406, 2005.

Digital Library

[2]

I. T. Arkin, H. Xu, K. J. Bowers, et al., Mechanism of a Na+/H+ Antiporter, submitted, 2006.

[3]

K. J. Bowers, Speed Optimal Implementation of a Fully Relativistic 3D Particle Push with a Charge Conserving Current Accumulate on Modern Processors, presented at 18th International Conference on the Numerical Simulation of Plasmas, Cape Cod, MA, 2003.

[4]

K. J. Bowers, R. O. Dror, and D. E. Shaw, Overview of Neutral Territory Methods for the Parallel Evaluation of Pairwise Particle Interactions, J. Phys. Conf. Ser., 16: 300--304, 2005.

[5]

K. J. Bowers, R. O. Dror, and D. E. Shaw, The Midpoint Method for Parallelization of Particle Simulations, J. Chem. Phys., 124: 184109, 2006.

[6]

K. J. Bowers, R. O. Dror, and D. E. Shaw, Zonal Methods for the Parallel Execution of Range-Limited N-Body Problems, in press, J. Comput. Phys., 2006.

Digital Library

[7]

B. R. Brooks, R. E. Bruccoleri, B. D. Olafson, et al., CHARMM: A Program for Macromolecular Energy, Minimization, and Dynamics Calculations, J. Comput. Chem., 4: 187--217, 1983.

[8]

C. L. Brooks, B. M. Pettit, and M. Karplus, Structural and Energetic Effects of Truncating Long Ranged Interactions in Ionic and Polar Fluids, J. Chem. Phys., 83(11): 5897--5908, 1985.

[9]

F. Cappello and D. Etiemble, MPI Versus MPI+OpenMP on the IBM SP for the NAS Benchmarks, presented at ACM/IEEE SC2000 Conference, Dallas, TX, 2000.

Digital Library

[10]

D. A. Case, T. E. Cheatham, III, T. Darden, et al., The Amber Biomolecular Simulation Programs, J. Comput. Chem., 26(16): 1668--1688, 2005.

[11]

E. Chow and D. Hysom, Assessing Performance of Hybrid MPI/OpenMP Programs on SMP Clusters, Lawrence Livermore National Laboratory UCRL-JC-143957, 2001.

[12]

T. Darden, D. York, and L. Pedersen, Particle Mesh Ewald: An N Log(N) Method for Ewald Sums in Large Systems, J. Chem. Phys., 98(12): 10089--10092, 1993.

[13]

Y. Duan and P. A. Kollman, Pathways to a Protein Folding Intermediate Observed in a 1-Microsecond Simulation in Aqueous Solution, Science, 282(5389): 740--744, 1998.

[14]

M. Eleftheriou, B. G. Fitch, A. Rayshubskiy, et al., Scalable Framework for 3D FFTs on the Blue Gene/L Supercomputer: Implementation and Early Performance Measurements, IBM J. Res. & Dev., 49(2-3): 457--464, 2005.

Digital Library

[15]

B. G. Fitch, A. Rayshubskiy, M. Eleftheriou, et al., Blue Matter: Strong Scaling of Molecular Dynamics on Blue Gene/L, IBM RC23888, February 22, 2006.

[16]

B. G. Fitch, A. Rayshubskiy, M. Eleftheriou, et al., Blue Matter: Approaching the Limits of Concurrency for Classical Molecular Dynamics, IBM RC23956, May 12, 2006.

[17]

B. G. Fitch, A. Rayshubskiy, M. Eleftheriou, et al., Blue Matter: Strong Scaling of Molecular Dynamics on Blue Gene/L, IBM RC23688, August 5, 2005.

[18]

M. Frigo and S. G. Johnson, The Design and Implementation of FFTW3, Proceedings of the IEEE, 93(2): 216--231, 2005.

[19]

R. S. Germain, B. Fitch, A. Rayshubskiy, et al., Blue Matter on Blue Gene/L: Massively Parallel Computation for Biomolecular Simulation, presented at 3rd IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis (CODES+ISSS'05), New York, NY, 2005.

Digital Library

[20]

T. A. Halgren, MMFF VII. Characterization of MMFF94, MMFF94s, and Other Widely Available Force Fields for Conformational Energies and for Intermolecular-Interaction Energies and Geometries, J. Comput. Chem., 20(7): 730--748, 1999.

[21]

G. S. Heffelfinger, Parallel Atomistic Simulations, Comput. Phys. Commun., 128(1-2): 219--237, 2000.

[22]

W. L. Jorgensen, D. S. Maxwell, and J. Tirado-Rives, Development and Testing of the OPLS All-Atom Force Field on Conformational Energetics and Properties of Organic Liquids, J. Am. Chem. Soc., 118(45): 11225--11236, 1996.

[23]

P. A. Kollman, R. W. Dixon, W. D. Cornell, et al., "The Development/Application of a "Minimalist" Organic/Biomolecular Mechanic Forcefield Using a Combination of Ab Initio Calculations and Experimental Data," in Computer Simulation of Biomolecular Systems: Theoretical and Experimental Applications, W. F. van Gunsteren and P. K. Weiner, Eds. Dordrecht, Netherlands: ESCOM, 1997, 83--96.

[24]

S. Kumar, G. Almasi, C. Huang, et al., Achieving Strong Scaling with NAMD on Blue Gene/L, presented at IEEE International Parallel & Distributed Processing Symposium, Rhodes Island, Greece, 2006.

Digital Library

[25]

J. Liu, J. Wu, and D. K. Panda, High Performance RDMA-Based MPI Implementation over InfiniBand, presented at 17th International Conference on Supercomputing, San Francisco, CA, 2003.

Digital Library

[26]

J. MacKerell, A. D., D. Bashford, M. Bellott, et al., All-Atom Empirical Potential for Molecular Modeling and Dynamics Studies of Proteins, J. Phys. Chem. B, 102(18): 3586--3616, 1998.

[27]

P. Mark and L. Nilsson, Structure and Dynamics of Liquid Water with Different Long-Range Interaction Truncation and Temperature Control Methods in Molecular Dynamics Simulations, J. Comput. Chem., 23(13): 1211--1219, 2002.

[28]

Mellanox Technologies, Mellanox IB-Verbs API (VAPI): Mellanox Software Programmer's Interface for InfiniBand Verbs, 2001.

[29]

T. Narumi, A. Kawai, and T. Koishi, An 8.61 Tflop/s Molecular Dynamics Simulation for NaCl with a Special-Purpose Computer: MDM, presented at ACM/IEEE SC2001 Conference, Denver, Colorado, 2001.

Digital Library

[30]

J. Norberg and L. Nilsson, On the Truncation of Long-Range Electrostatic Interactions in DNA, Biophys. J., 79(3): 1537--1553, 2000.

[31]

V. S. Pande, I. Baker, J. Chapman, et al., Atomistic Protein Folding Simulations on the Submillisecond Time Scale Using Worldwide Distributed Computing, Biopolymers, 68(1): 91--109, 2003.

[32]

P. M. Papadopoulos, M. J. Katz, and G. Bruno, NPACI Rocks: Tools and Techniques for Easily Deploying Manageable Linux Clusters, Concurrency Comput. Pract. Ex., 15(7-8): 707--725, 2003.

[33]

M. Patra, M. Karttunen, T. Hyvönen, et al., Molecular Dynamics Simulations of Lipid Bilayers: Major Artifacts Due to Truncating Electrostatic Interactions, Biophys. J., 84: 3636--3645, 2003.

[34]

J. C. Phillips, R. Braun, W. Wang, et al., Scalable Molecular Dynamics with NAMD, J. Comput. Chem., 26(16): 1781--1802, 2005.

[35]

J. C. Phillips, G. Zheng, S. Kumar, et al., NAMD: Biomolecular Simulation on Thousands of Processors, presented at ACM/IEEE SC2002 Conference, Baltimore, 2002.

Digital Library

[36]

S. Plimpton, Fast Parallel Algorithms for Short-Range Molecular-Dynamics, J. Comput. Phys., 117(1): 1--19, 1995.

Digital Library

[37]

S. Plimpton and B. Hendrickson, Parallel Molecular-Dynamics Simulations of Organic Materials, Int. J. Mod. Phys. C., 5(2): 295--298, 1994.

[38]

S. Plimpton and B. Hendrickson, A New Parallel Method for Molecular Dynamics Simulation of Macromolecular Systems, J. Comput. Chem., 17(3): 326--337, 1996.

[39]

W. R. P. Scott, P. H. Hünenberger, I. G. Tironi, et al., The GROMOS Biomolecular Simulation Program Package, J. Phys. Chem. A, 103(19): 3596--3607, 1999.

[40]

M. M. Seibert, A. Patriksson, B. Hess, et al., Reproducible Polypeptide Folding and Structure Prediction Using Molecular Dynamics Simulations, J. Mol. Biol., 354(1): 173--183, 2005.

[41]

Y. Shan, J. L. Klepeis, M. P. Eastwood, et al., Gaussian Split Ewald: A Fast Ewald Mesh Method for Molecular Simulation, J. Chem. Phys., 122: 054101, 2005.

[42]

T. Shanley, InfiniBand Network Architecture. Boston: Addison-Wesley, 2003.

Digital Library

[43]

D. E. Shaw, A Fast, Scalable Method for the Parallel Evaluation of Distance-Limited Pairwise Particle Interactions, J. Comput. Chem., 26(13): 1318--1328, 2005.

[44]

M. Snir, A Note on N-Body Computations with Cutoffs, Theor. Comput. Syst., 37: 295--318, 2004.

[45]

D. van der Spoel, E. Lindahl, B. Hess, et al., GROMACS: Fast, Flexible, and Free, Journal of Computational Chemistry, 26(16): 1701--1718, 2005.

[46]

M. Taiji, T. Narumi, Y. Ohno, et al., Protein Explorer: A Petaflops Special-Purpose Computer System for Molecular Dynamics Simulations, presented at ACM/IEEE SC2003 Conference, Phoenix, Arizona, 2003.

Digital Library

[47]

R. Zhou and B. J. Berne, A New Molecular Dynamics Method Combining the Reference System Propagator Algorithm with a Fast Multipole Method for Simulating Proteins and Other Complex Systems, J. Chem. Phys., 103(21): 9444--9459, 1995.

[48]

R. Zhou, E. Harder, H. Xu, et al., Efficient Multiple Time Step Method for Use with Ewald and Particle Mesh Ewald for Large Biomolecular Systems, J. Chem. Phys., 115(5): 2348--2358, 2001.

Cited By

Islam MHa MMin BChoi JJung H(2025)Unveiling the Multitarget Potential of a Rare Caffeoyl Ester from Artemisia capillaris for Diabetes Mellitus: An Integrated In Vitro and In Silico StudyInternational Journal of Molecular Sciences10.3390/ijms2603128626:3(1286)Online publication date: 2-Feb-2025
https://doi.org/10.3390/ijms26031286
V. PA. MN. R(2025)A Computational Study of Phenothiazine Derivatives as Acetylcholinesterase Inhibitors Targeting Alzheimer’s DiseaseCentral Nervous System Agents in Medicinal Chemistry10.2174/011871524930078424043011062825:1(68-82)Online publication date: Jan-2025
https://doi.org/10.2174/0118715249300784240430110628
Al Mashud MDevnath RAnzuman MSumona MHossain MKumer ATalukder MRahman MImon RAkash SEl Moussaoui ASalamatullah ABourhia M(2025)New Approach as Inhibitor Against Head-Neck Cancer by In silico, DFT, FMOs, Docking, Molecular Dynamic, and ADMET of Euphorbia tirucalli (Pencil Cactus)Medicinal Chemistry10.2174/011573406431560124062811533021:2(122-143)Online publication date: Feb-2025
https://doi.org/10.2174/0115734064315601240628115330
Show More Cited By

Index Terms

Recommendations

MPPs and clusters for scalable computing
ISPAN '96: Proceedings of the 1996 International Symposium on Parallel Architectures, Algorithms and Networks

This article assess the state-of-the-art technology in massively parallel processors (MPPs) and clusters of workstations (COWs) for scalable parallel computing. We evaluate the IBM SP2, the Intel Paragon, the Cray T3D/T3E, and the ASCI TeraFLOPS system ...
Massively parallel molecular dynamics simulations of lysozyme unfolding

We have performed molecular dynamics simulations for a total duration of more than 10 µs (with most molecular trajectories being 1 µs in duration) to study, the effect of a single mutation on hen lysozyme protein stability and denaturing, using an IBM ...
Identification of novel PI3Kδ inhibitors by docking, ADMET prediction and molecular dynamics simulations
Graphical abstract
The virtual screening, molecular docking, ADMET prediction and molecular dynamics simulations were performed on Asinex database to find the novel compounds against PI3Kδ, which were demonstrated as the following flow chart:
...
Highlights
- Phosphoinositide-3-kinase Delta (PI3Kδ) is a well-validated target for anticancer drug design and development.
Abstract Background
Phosphoinositide-3-kinase Delta (PI3Kδ) plays a key role in B-cell signal transduction and inhibition of PI3Kδ is confirmed to have clinical benefit in certain types of activation of B-cell malignancies. Virtual ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '06: Proceedings of the 2006 ACM/IEEE conference on Supercomputing

November 2006

746 pages

ISBN:0769527000

DOI:10.1145/1188455

Conference Chair:
Barbara Horner-Miller
Arctic Region Supercomputing Center

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

SC '06

Sponsor:

SIGARCH
IEEE-CS

SC '06: International Conference for High Performance Computing, Networking, Storage and Analysis

November 11 - 17, 2006

Florida, Tampa

Acceptance Rates

SC '06 Paper Acceptance Rate 54 of 239 submissions, 23%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1,406
Total Citations
View Citations
2,650
Total Downloads

Downloads (Last 12 months)312
Downloads (Last 6 weeks)30

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Islam MHa MMin BChoi JJung H(2025)Unveiling the Multitarget Potential of a Rare Caffeoyl Ester from Artemisia capillaris for Diabetes Mellitus: An Integrated In Vitro and In Silico StudyInternational Journal of Molecular Sciences10.3390/ijms2603128626:3(1286)Online publication date: 2-Feb-2025
https://doi.org/10.3390/ijms26031286
V. PA. MN. R(2025)A Computational Study of Phenothiazine Derivatives as Acetylcholinesterase Inhibitors Targeting Alzheimer’s DiseaseCentral Nervous System Agents in Medicinal Chemistry10.2174/011871524930078424043011062825:1(68-82)Online publication date: Jan-2025
https://doi.org/10.2174/0118715249300784240430110628
Al Mashud MDevnath RAnzuman MSumona MHossain MKumer ATalukder MRahman MImon RAkash SEl Moussaoui ASalamatullah ABourhia M(2025)New Approach as Inhibitor Against Head-Neck Cancer by In silico, DFT, FMOs, Docking, Molecular Dynamic, and ADMET of Euphorbia tirucalli (Pencil Cactus)Medicinal Chemistry10.2174/011573406431560124062811533021:2(122-143)Online publication date: Feb-2025
https://doi.org/10.2174/0115734064315601240628115330
Sharma ANarang AKumar NRana RMegha Pooja Dhir MGulati HJyoti Khanna ASingh JKaur SBedi P(2025)CADD based designing and biological evaluation of novel triazole based thiazolidinedione coumarin hybrids as antidiabetic agentScientific Reports10.1038/s41598-025-88944-y15:1Online publication date: 4-Feb-2025
https://doi.org/10.1038/s41598-025-88944-y
El-Daly MBajrai LAlandijany TAlsaady IGattan HAlhamdan MDwivedi VAzhar E(2025)Exploring Echinacea angustifolia for anti-viral compounds against Zika virus RNA-dependent RNA polymerase: a computational studyScientific Reports10.1038/s41598-025-88481-815:1Online publication date: 3-Feb-2025
https://doi.org/10.1038/s41598-025-88481-8
Syed RMoni SHussein WAlhaidan TAbumilha SAlnahdi LWong LSubramaniyan VKumarasamy V(2025)Effect of cubebin against streptozotocin-induced diabetic nephropathy rats via inhibition TNF-α/NF-κB/TGF-β: in vivo and in silico studyScientific Reports10.1038/s41598-025-87319-715:1Online publication date: 5-Feb-2025
https://doi.org/10.1038/s41598-025-87319-7
Alharbi KAfzal MAl-Abbasi FMoglad EAl-Qahtani SAlmalki NImam FSayyed NKazmi I(2025)In vivo and in silico study of europinidin against streptozotocin-isoproterenol-induced myocardial damage via alteration of hs-CRP/CPK-MB/Caspase-3/Bcl-2 pathwaysScientific Reports10.1038/s41598-024-83900-815:1Online publication date: 24-Jan-2025
https://doi.org/10.1038/s41598-024-83900-8
Arshad FSarfraz AShehroz MNishan UPerveen AUllah RIbrahim MShah M(2025)Core-genome guided novel therapeutic targets identification and chimeric vaccine designing against Rickettsia rickettsiiScientific Reports10.1038/s41598-024-83395-315:1Online publication date: 6-Jan-2025
https://doi.org/10.1038/s41598-024-83395-3
Parveen SIkhlas SFaruqui THasan AKhatoon ASaeed MAlkhathami ASiddiqui SUddin SMir S(2025)Identifying Potential Autophagy Modulators in Panch Phoron Spices (P5S): An In Silico approachACS Omega10.1021/acsomega.4c07924Online publication date: 2-Jan-2025
https://doi.org/10.1021/acsomega.4c07924
Rajak BRani PSingh NSingh D(2025) In Silico Evaluation and Simulation-Based Prioritization of Herbicide-like Compounds Targeting Phalaris minor Acetyl-CoA Carboxylase ACS Agricultural Science & Technology10.1021/acsagscitech.4c00635Online publication date: 17-Jan-2025
https://doi.org/10.1021/acsagscitech.4c00635
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten