research-article

Free access

How will astronomy archives survive the data tsunami?

Authors:

G. Bruce Berriman,

Steven L. GroomAuthors Info & Claims

Communications of the ACM, Volume 54, Issue 12

Pages 52 - 56

https://doi.org/10.1145/2043174.2043190

Published: 01 December 2011 Publication History

All formats PDF

Abstract

Astronomers are collecting more data than ever. What practices can keep them ahead of the flood?

References

[1]

Barsdell, B.R., Barnes, D.G. and Fluke, C. J. Analysing astronomy algorithms for graphics processing units and beyond. Monthly Notices of the Royal Astronomical Society 408, 3 (2010), 1936--1944.

Crossref

Google Scholar

[2]

Berriman, G.B., Deelman, E., Juve, G., Regelson, M. and Plavchan, P. The application of cloud computing to astronomy: A study of cost and performance. Accepted for publication in Proceedings of the e-Science in Astronomy Conference (Brisbane, AU, 2011).

Digital Library

Google Scholar

[3]

Berriman, G.B., Good, J., Deelman, E. and Alexov, A. Ten years of software sustainability at the Infrared Processing and Analysis Center. Phil. Trans. R. Soc. 369 (2011), 3384--3397

Crossref

Google Scholar

[4]

Canon, S. Debunking some common misconceptions of science in the cloud. Presented at ScienceCloud2011: 2nd Workshop on Scientific Cloud Computing (San Jose, CA); http://datasys.cs.iit.edu/events/ScienceCloud2011/.

Google Scholar

[5]

Chilingarian, I. and Zolotukhin, I. The true bottleneck of modern scientific computing in astronomy. Astronomical Data Analysis Software and Systems XX. I. Evans et al., Eds. ASP Conference Series 442, 471 (2010).

Google Scholar

[6]

Committee for a Decadal Survey of Astronomy and Astrophysics, National Research Council of the National Academy of Sciences. New Worlds, New Horizons in Astronomy and Astrophysics, 2010.

Google Scholar

[7]

Fluke, C.J., Barnes, D.G., Barsdell, B.R., Hassan, A.H. Astrophysical supercomputing with GPUs: critical decisions for early adopters. Publications of the Astronomical Society of Australia 28, 15 (2011).

Crossref

Google Scholar

[8]

Gaudet, S. et al. CANFAR: The Canadian Advanced Network for Astronomical Research. Software and Cyber Infrastructure for Astronomy. N. Radziwill and A. Bridger, Eds. SPIE 7740, 1I (2010).

Google Scholar

[9]

Good, J. Private communication, 2011.

Google Scholar

[10]

Hanisch, R.J. Data discovery, access, and management with the virtual observatory. 2011, Paper presented at Innovations in Data-intensive Astronomy, Green Bank, WV, 2011; http://www.nrao.edu/meetings/bigdata/presentations/May5/1-Hanisch/Hanisch VAO Green Bank.ppt.

Google Scholar

[11]

Hemsoth, N. Canada explores new frontiers in astroinformatics. HPC in the Cloud; http://www.hpcinthecloud.com/hpccloud/2011-01-17/canada_explores_new_frontiers_in_astroinformatics.html.

Google Scholar

[12]

Hutchinson, J. SKA bid looks to SkyNet for computing, 2011; http://www.cio.com.au/article/387097/exclusive_ska_bid_looks_skynet_computing/.

Google Scholar

[13]

Loffler, F., Allen, G., Benger, W., Hutanu, A., Jha, S. and Schnetter, E. Using the TeraGrid to teach scientific computing. TeraGrid '11: Extreme Digital Discovery Conference (Salt Lake City, UT; July 18--21, 2011); https://www.teragrid.org/web/tg11/home.

Digital Library

Google Scholar

[14]

Merali, Z. Why scientific programming does not compute. Nature 467 (2010) 775--777.

Crossref

Google Scholar

[15]

National Radio Astronomy Observatory. Innovations in Data-intensive Astronomy Workshop (Green Bank, WV, May 3--5, 2011); http://www.nrao.edu/meetings/bigdata/index.shtml.

Google Scholar

[16]

SciDB Open Source Data Management and Analytics Software System. 2011; http://scidb.org.

Google Scholar

[17]

Software Carpentry; http://software-carpentry.org/.

Google Scholar

[18]

Space Telescope Science Institute. Hubble Space Telescope Publication Statistics 1991--2010 (2011); http://archive.stsci.edu/hst/bibliography/pubstat.html.

Google Scholar

[19]

Virtual Astronomical Observatory; http://us-vao.org.

Google Scholar

[20]

Nereus overview. http://www-nereus.physics.ox.ac.uk/about_overview.html

Google Scholar

Cited By

View all

Sen SAgarwal SChakraborty PSingh K(2022)Astronomical big data processing using machine learning: A comprehensive reviewExperimental Astronomy10.1007/s10686-021-09827-453:1(1-43)Online publication date: 14-Jan-2022
https://doi.org/10.1007/s10686-021-09827-4
Barrientos AHoldship JSolar MMartín SRivilla VViti SMangum JHarada NSakamoto KMuller STanaka KYoshimura YNakanishi KHerrero-Illana RMühle SAladro RAalto SHenkel CHumire P(2021)Towards the prediction of molecular parameters from astronomical emission lines using Neural NetworksExperimental Astronomy10.1007/s10686-021-09786-wOnline publication date: 18-Sep-2021
https://doi.org/10.1007/s10686-021-09786-w
Fluke CHegarty SMacMahon C(2020)Understanding the human in the design of cyber-human discovery systems for data-driven astronomyAstronomy and Computing10.1016/j.ascom.2020.10042333(100423)Online publication date: Oct-2020
https://doi.org/10.1016/j.ascom.2020.100423
Show More Cited By

Index Terms

How will astronomy archives survive the data tsunami?
1. Applied computing
  1. Physical sciences and engineering
2. Information systems

Recommendations

How Will Astronomy Archives Survive the Data Tsunami?: Astronomers are collecting more data than ever. What practices can keep them ahead of the flood?
Programming Languages

Astronomy is already awash with data: currently 1 PB of public data is electronically accessible, and this volume is growing at 0.5 PB per year. The availability of this data has already transformed research in astronomy, and the STScI now reports that ...
How To Survive Y2k Chaos In The City: A Preparedness and Self-Reliance Handbook
Taming the Information Tsunami

Comments

Information & Contributors

Information

Published In

Communications of the ACM Volume 54, Issue 12

December 2011

121 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/2043174

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2011

Published in CACM Volume 54, Issue 12

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Popular
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
25,797
Total Downloads

Downloads (Last 12 months)2,012
Downloads (Last 6 weeks)10

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Sen SAgarwal SChakraborty PSingh K(2022)Astronomical big data processing using machine learning: A comprehensive reviewExperimental Astronomy10.1007/s10686-021-09827-453:1(1-43)Online publication date: 14-Jan-2022
https://doi.org/10.1007/s10686-021-09827-4
Barrientos AHoldship JSolar MMartín SRivilla VViti SMangum JHarada NSakamoto KMuller STanaka KYoshimura YNakanishi KHerrero-Illana RMühle SAladro RAalto SHenkel CHumire P(2021)Towards the prediction of molecular parameters from astronomical emission lines using Neural NetworksExperimental Astronomy10.1007/s10686-021-09786-wOnline publication date: 18-Sep-2021
https://doi.org/10.1007/s10686-021-09786-w
Fluke CHegarty SMacMahon C(2020)Understanding the human in the design of cyber-human discovery systems for data-driven astronomyAstronomy and Computing10.1016/j.ascom.2020.10042333(100423)Online publication date: Oct-2020
https://doi.org/10.1016/j.ascom.2020.100423
Lee JCanon RDeclerck TDraney BPaul DSkinner D(2018)Enhancing supercomputing with software defined networking2018 International Conference on Information Networking (ICOIN)10.1109/ICOIN.2018.8343183(571-576)Online publication date: Jan-2018
https://doi.org/10.1109/ICOIN.2018.8343183
Farias HNuñez CSolar M(2018)TensorFit a tool to analyse spectral cubes in a tensor modeAstronomy and Computing10.1016/j.ascom.2018.10.00725(195-202)Online publication date: Oct-2018
https://doi.org/10.1016/j.ascom.2018.10.007
Bertocco SDowler PGaudet SMajor BPasian FTaffoni G(2018)Cloud access to interoperable IVOA-compliant VOSpace storageAstronomy and Computing10.1016/j.ascom.2018.05.00324(36-44)Online publication date: Jul-2018
https://doi.org/10.1016/j.ascom.2018.05.003
Meade BFluke C(2018)Evaluating virtual hosted desktops for graphics-intensive astronomyAstronomy and Computing10.1016/j.ascom.2018.04.00223(124-140)Online publication date: Apr-2018
https://doi.org/10.1016/j.ascom.2018.04.002
Li BYu CHu XXiao JTang SLi LMa B(2018)An Efficient Retrieval Method for Astronomical Catalog Time Series DataAlgorithms and Architectures for Parallel Processing10.1007/978-3-030-05051-1_20(284-298)Online publication date: 7-Dec-2018
https://doi.org/10.1007/978-3-030-05051-1_20
Rheinländer ALeser UGraefe G(2017)Optimization of Complex Dataflows with User-Defined FunctionsACM Computing Surveys10.1145/307875250:3(1-39)Online publication date: 26-May-2017
https://dl.acm.org/doi/10.1145/3078752
Cranmer MBarsdell BPrice DDowell JGarsden HDike VEftekhari THegedus AMalins JObenberger KSchinzel FStovall KTaylor GGreenhill L(2017)Bifrost: A Python/C++ Framework for High-Throughput Stream Processing in AstronomyJournal of Astronomical Instrumentation10.1142/S225117171750007606:04Online publication date: 27-Dec-2017
https://doi.org/10.1142/S2251171717500076
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations