Article

Free access

Automatic parallel I/O performance optimization in Panda

Authors:

Y. Chen,

M. Winslett,

Y. Cho,

S. KuoAuthors Info & Claims

SPAA '98: Proceedings of the tenth annual ACM symposium on Parallel algorithms and architectures

Pages 108 - 118

https://doi.org/10.1145/277651.277677

Published: 01 June 1998 Publication History

PDF eReader

References

[1]

J. M. Anderson, S. P. Amarasinghe, and M. S. Lain. Data and comtmtation transformations for multii)rocessors. In Proceedings of the Fifth A CM SIG- PLAN Sympo.sium on Principles and Practice of Parallel Processing, 1995.

Digital Library

Google Scholar

[2]

R. Bordawekar, A. Choudhary, and J. Ramanujam. Automatic ()ptimization of communication in com- I)iling out-()f-c()rc stencil ('()(los. In Procceding,s of the iOth, A CM International Conference on Supercomputing, pages 366-373, Philadelphia, PA. May 1996. ACM Press.

Crossref

Google Scholar

[3]

E. B()r()wsky, R. Gol ling, A. Merchant. E. Shriver, M. Sl)as()jevic. an(1 J. Wilkes. Eliininating storage headaches through self-management. In Procecding.s oJ th, c Second Sympo.#ium on Operating Sy.#tem.s Design and Implementation. Scatth#', \VA, 1996.

Google Scholar

[4]

M.J. Box, D. Davies. and \V.H. Swarm. Non-linear ()ptimization t(,chniques, ht ICI Monograph No. 5; Ediid)urgh. 1969. Oliver & Boyd.

Google Scholar

[5]

Y. Chcn. I. Foster, J. Ni(:t)l()(:ha, an(t M. \:Vinslctt. Optimizing c#ollcctivc I/O I)crfl)rmanee on t)aralM COmlmters: A multisystem study, ht Proceedings of 11th A CM International Conference on Supercomp',.ting, pages 28 35. ACM Press. Jltly 1997.

Digital Library

Google Scholar

[6]

Y. Chert, M. \Vinslett, Y. Ch(), and S. Kuo. Speeding up autoinatic I)arallel I/O performan(:c ot)timization in Panda. In Proceedings of the 12th. Av,',ual International Sympo,#i',m on High Perfof mance Computing Systcm,s and Applications, Edmont()n. Cana(la. May 1998. Kluwer Aca(temic Pul)lishers.

Google Scholar

[7]

Y. Chen, M. \.ViIlslctt, S. Ku(), Y. Cho, M. Sul)ramaniam, and K.E. Seaim)ns. PerforInance modeling for the Pan(ta array I/O library, in Proceedings o.f Supercomputing '96. ACM Press and IEEE Comt)#ltcr Society Press, November 1996.

Digital Library

Google Scholar

[8]

D.E. Gohtberg. Genetic algorithm,# in search, optimization and machine lea','ning. A(htison-\V(Mcy, Reading, MA, 1989.

Digital Library

Google Scholar

[9]

G. Graefe. Query ewduation techniques for large databases. Computing Surveys, 25(2)'73-170, 1993.

Digital Library

Google Scholar

[10]

R. Hooke and T.A. Jccves. ':Direct Search" solution of numerical and statistical l)rot)lems. Journal of the A.ssociation for Computing Machinery, 8:212- 229, 1961.

Digital Library

Google Scholar

[11]

J. Hut)er, C.L. Elfor(1, D.A. Roe(l, A.A. Chicn, an(l D.S. Blumenthal. P PFS" A high l)erformancc l)()rtable parallel ill(', system. In Proceedings of the 9th, ACM International Conference on Supercomputin, g, pages 385 394. Barc(,q()na. July 1995. ACM Press.

Digital Library

Google Scholar

[12]

L. Ingbcr. Simulatc(t annealing: Pra('ti(:c vcrsns thcorv. Mathl. Compu. Modelling, 18(11):29 57, 1993.

Google Scholar

[13]

Y. Ioannidis and E. Wong. Query optimizatioil by simulated annealing. In Proceedings of the 1987 A CM-SIGMOD International Conference on Management of Data, pages 9-.22, 1987.

Digital Library

Google Scholar

[14]

S. Kirkpatriek, C.D. Gelatt Jr., and M.P. Vecchi. Optimization by simulated annealing. Science, 220(4598):671-680, 1983.

Crossref

Google Scholar

[15]

D. Kotz an(t N. Nieuwejaar. Dynamic file-access characteristics ()f a l)roduction parallel scientifi(: workload. In Proceedings of Supercornputing '94, i)agcs 640-649, Washington, DC, Noveml)er 1994. IEEE ComI)utcr $o('iety Press.

Digital Library

Google Scholar

[16]

T.M. Madhyasta, C.L. Elford, an(l D.A. Reed. Optimizing input/output using adaptive file system p()li(:ics. In Proceedings of the Fifth NASA Goddard Conference on Mass Storage System.s, t)ages II:493-514, Sel)tember 1996.

Google Scholar

[17]

T.M. Ma(lhyasta and D.A. Reed. Intelligent, adat)- tiw# file system policy selection. In Proceeding.s of th, e Sixth, Sympo,sium on the Frontier,s of Massively Parallel Corn, putation, pages 172 179. IEEE Computer So('icty Press, October 1996.

Crossref

Google Scholar

[18]

J. Ni(3)h)(:ha and I. F()ster. Disk resident arrays: An array-()ri(#ide(t 1/O library for out-of-core comt)utations. In Proceedings of the Sixth Sympo,sium on the Frontiers of Ma,ssively Parallel Computation, pages 196 204. IEEE Computer Society Press, Octot)er 1996.

Crossref

Google Scholar

[19]

R. Hugo Patterson, G.A. Gibson, Ekat Ginting, Daniel Stodolsky, and Jim Zelenka. Informed prefetching and caching. In Proceedings of the Fifteenth, A CM Symposium on Operating Sy.stem#s Principles, pages 79-95, Copper Mountain, CO, December 1995. ACM Press.

Digital Library

Google Scholar

[20]

K.E. Seamons, Y. Chen, P. Jones, J. J()zwiak, and M. Winslett. Server-dirccte(l collective I/O in Panda. In Proceedings of Supercomputing '95, San Diego, CA, December 1995. IEEE Computer Society Press.

Digital Library

Google Scholar

[21]

A. Swami and A. Gupta. Optimization of large join queries. In Proceedings of the 1988 A CM-SIGMOD international Conference on Management of Data, pages 8-17, Chicago, IL, 1988.

Digital Library

Google Scholar

[22]

R. Thakur, A. Choudhary, R. Bordawekar, S. More. and S. KuditiI)udi. PASSION: Optimized I/O for l)arallcl at)plications. IEEE Computer, 29(6):70-78, June 1996.

Digital Library

Google Scholar

Cited By

View all

Liu ZZhang CWu HFang JPeng LYe GTang Z(2023)Optimizing HPC I/O Performance with Regression Analysis and Ensemble Learning2023 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER52292.2023.00027(234-246)Online publication date: 31-Oct-2023
https://doi.org/10.1109/CLUSTER52292.2023.00027
Behzad BByna SPrabhat Snir M(2019)Optimizing I/O Performance of HPC Applications with AutotuningACM Transactions on Parallel Computing10.1145/33092055:4(1-27)Online publication date: 8-Mar-2019
https://dl.acm.org/doi/10.1145/3309205
Ji YYu CXiao JTang SWang HZhang B(2019)HDF5-Based I/O Optimization for Extragalactic HI Data Pipeline of FASTAlgorithms and Architectures for Parallel Processing10.1007/978-3-030-38961-1_55(656-672)Online publication date: 9-Dec-2019
https://dl.acm.org/doi/10.1007/978-3-030-38961-1_55
Show More Cited By

Index Terms

Automatic parallel I/O performance optimization in Panda
1. Software and its engineering
  1. Software creation and management
    1. Designing software
  2. Software organization and properties
    1. Contextual software domains
      1. Operating systems

Recommendations

Comments

Information & Contributors

Information

Published In

SPAA '98: Proceedings of the tenth annual ACM symposium on Parallel algorithms and architectures

June 1998

312 pages

ISBN:0897919890

DOI:10.1145/277651

Chairmen:
Gary Miller
Carnegie Mellon Univ., Pittsburgh, PA
,
Phillip B. Gibbons
Bell Labs

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 1998

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

SPAA/PODC98

Sponsor:

SPAA/PODC98: 10th Annual ACM Symposium on Parallel Algorithms and Architectures/Symposium on Principles of Distributed Computing

June 28 - July 2, 1998

Puerto Vallarta, Mexico

Acceptance Rates

SPAA '98 Paper Acceptance Rate 30 of 84 submissions, 36%;

Overall Acceptance Rate 447 of 1,461 submissions, 31%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
595
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)2

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Liu ZZhang CWu HFang JPeng LYe GTang Z(2023)Optimizing HPC I/O Performance with Regression Analysis and Ensemble Learning2023 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER52292.2023.00027(234-246)Online publication date: 31-Oct-2023
https://doi.org/10.1109/CLUSTER52292.2023.00027
Behzad BByna SPrabhat Snir M(2019)Optimizing I/O Performance of HPC Applications with AutotuningACM Transactions on Parallel Computing10.1145/33092055:4(1-27)Online publication date: 8-Mar-2019
https://dl.acm.org/doi/10.1145/3309205
Ji YYu CXiao JTang SWang HZhang B(2019)HDF5-Based I/O Optimization for Extragalactic HI Data Pipeline of FASTAlgorithms and Architectures for Parallel Processing10.1007/978-3-030-38961-1_55(656-672)Online publication date: 9-Dec-2019
https://dl.acm.org/doi/10.1007/978-3-030-38961-1_55
Behzad BLuu HHuchette JByna SPrabhat Aydt RKoziol QSnir MGropp WMatsuoka S(2013)Taming parallel I/O complexity with auto-tuningProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.1145/2503210.2503278(1-12)Online publication date: 17-Nov-2013
https://dl.acm.org/doi/10.1145/2503210.2503278
Lin KByna SChou JWu K(2013)Optimizing fastquery performance on lustre file systemProceedings of the 25th International Conference on Scientific and Statistical Database Management10.1145/2484838.2484853(1-12)Online publication date: 29-Jul-2013
https://dl.acm.org/doi/10.1145/2484838.2484853
Manzanares ABent JWingate MGibson G(2012)The Power and Challenges of Transformative I/OProceedings of the 2012 IEEE International Conference on Cluster Computing10.1109/CLUSTER.2012.86(144-154)Online publication date: 24-Sep-2012
https://dl.acm.org/doi/10.1109/CLUSTER.2012.86
Kandemir MSon SKarakoy M(2008)Improving I/O performance of applications through compiler-directed code restructuringProceedings of the 6th USENIX Conference on File and Storage Technologies10.5555/1364813.1364824(1-16)Online publication date: 26-Feb-2008
https://dl.acm.org/doi/10.5555/1364813.1364824
Seelam SKerstens ATeller P(2007)Throttling I/O streams to accelerate file-IO performanceProceedings of the Third international conference on High Performance Computing and Communications10.5555/2401945.2402024(718-731)Online publication date: 26-Sep-2007
https://dl.acm.org/doi/10.5555/2401945.2402024
Krishnan SKrishnamoorthy SBaumgartner GLam CRamanujam JSadayappan PChoppella V(2006)Efficient synthesis of out-of-core algorithms using a nonlinear optimization solverJournal of Parallel and Distributed Computing10.1016/j.jpdc.2005.06.01766:5(659-673)Online publication date: 1-May-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2005.06.017
Gao XSahoo SLam CRamanujam JLu QBaumgartner GSadayappan PPingali KYelick KGrimshaw A(2005)Performance modeling and optimization of parallel out-of-core tensor contractionsProceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming10.1145/1065944.1065980(266-276)Online publication date: 15-Jun-2005
https://dl.acm.org/doi/10.1145/1065944.1065980
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Automatic Parallel I/O Performance Optimization in Panda

Automatic parallel input/output performance optimization in panda

Automatic Performance Prediction of Parallel Programs