research-article

Free access

Big data and its technical challenges

Authors:

H. V. Jagadish,

Johannes Gehrke,

Alexandros Labrinidis,

Yannis Papakonstantinou,

Jignesh M. Patel,

Raghu Ramakrishnan,

Cyrus ShahabiAuthors Info & Claims

Communications of the ACM, Volume 57, Issue 7

Pages 86 - 94

https://doi.org/10.1145/2611567

Published: 01 July 2014 Publication History

All formats PDF

Abstract

Exploring the inherent technical challenges in realizing the potential of Big Data.

References

[1]

Computing Community Consortium. Advancing Discovery in Science and Engineering. Spring 2011.

Google Scholar

[2]

Computing Community Consortium. Advancing Personalized Education. Spring 2011.

Google Scholar

[3]

Computing Community Consortium. Smart Health and Wellbeing. Spring 2011.

Google Scholar

[4]

Computing Community Consortium. A Sustainable Future. Summer 2011.

Google Scholar

[5]

Computer Research Association. Challenges and Opportunities with Big Data. Community white paper available at http://cra.org/ccc/docs/init/bigdatawhitepaper.pdf

Google Scholar

[6]

Dobbie, W. and Fryer, Jr. R.G. Getting Beneath the Veil of Effective Schools: Evidence from New York City. NBER Working Paper No. 17632. Issued Dec. 2011.

Google Scholar

[7]

Economist. Drowning in numbers: Digital data will flood the planet---and help us understand it better. (Nov 18, 2011); http://www.economist.com/blogs/dailychart/2011/11/big-data-0

Google Scholar

[8]

Flood, M., Jagadish, H.V., Kyle, A., Olken, F. and Raschid, L. Using data for systemic financial risk management. In Proc. 5^th Biennial Conf. Innovative Data Systems Research (Jan. 2011).

Google Scholar

[9]

Forbes. Data-driven: Improving business and society through data. (Feb. 10, 2012); http://www.forbes.com/special-report/data-driven.html

Google Scholar

[10]

Gartner Group. Pattern-Based Strategy: Getting Value from Big Data. (July 2011 press release); http://www.gartner.com/it/page.jsp?id=1731916

Google Scholar

[11]

González, M.C., Hidalgo, C.A. and Barabási, A-L. Understanding individual human mobility patterns. Nature 453, (June 5, 2008), 779--782.

Crossref

Google Scholar

[12]

Hey, T., Tansley, S. and Tolle, K., eds. The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, 2009.

Google Scholar

[13]

Kahn, S.D. On the future of genomic data. Science 331, 6018 (Feb. 11, 2011), 728--729.

Crossref

Google Scholar

[14]

Lazar, D. et al. Computational social science. Science 323, 5915 (Feb. 6, 2009), 721--723.

Google Scholar

[15]

Lohr, A. The age of Big Data. New York Times (Feb. 11, 2012); http://www.nytimes.com/2012/02/12/sunday-review/big-datas-impact-in-the-world.html

Google Scholar

[16]

Lohr, S. How Big Data became so big. New York Times (Aug. 11, 2012); http://www.nytimes.com/2012/08/12/business/how-big-data-became-so-big-unboxed.html

Google Scholar

[17]

Manyika, J. et al. Big Data: The next frontier for innovation, competition, and productivity. McKinsey Global Institute. May 2011.

Google Scholar

[18]

National Science and Technology Council. Materials Genome Initiative for Global Competitiveness. June 2011.

Google Scholar

[19]

Noguchi, Y. Following the Breadcrumbs to Big Data Gold. National Public Radio (Nov. 29, 2011); http://www.npr.org/2011/11/29/142521910/the-digital-breadcrumbs-that-lead-to-big-data

Google Scholar

[20]

Noguchi, Y. The Search for Analysts to Make Sense of Big Data. National Public Radio, (Nov. 30, 2011); http://www.npr.org/2011/11/30/142893065/the-search-for-analysts-to-make-sense-of-big-data

Google Scholar

[21]

Pariser, E. The Filter Bubble: What the Internet Is Hiding From You. Penguin Press, May 2011.

Digital Library

Google Scholar

[22]

PCAST Report. Designing a Digital Future: Federally Funded Research and Development in Networking and Information Technology (Dec. 2010); http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-nitrd-report-2010.pdf

Google Scholar

[23]

SDSS-III: Massive Spectroscopic Surveys of the Distant Universe, the Milky Way Galaxy, and Extra-Solar Planetary Systems (Jan. 2008); http://www.sdss3.org/collaboration/description.pdf/

Google Scholar

Cited By

View all

AYISI LARBI COSEİ S(2024)Reduction of Losses and Wastage in Seafoods: The Role of Smart Tools and Biosensors Based on Artificial IntelligenceReduction of Losses and Wastage in Seafoods: The Role of Smart Tools and Biosensors Based on Artificial IntelligenceJournal of AI10.61969/jai.1394542Online publication date: 19-Feb-2024
https://doi.org/10.61969/jai.1394542
Zhang LZhu JJin BWei X(2024)Multiview Spatial-Temporal Meta-Learning for Multivariate Time Series ForecastingSensors10.3390/s2414447324:14(4473)Online publication date: 10-Jul-2024
https://doi.org/10.3390/s24144473
Sun YChen CXu YXie SBlum RVenkitasubramaniam P(2024)On the generalization discrepancy of spatiotemporal dynamics-informed graph convolutional networksFrontiers in Mechanical Engineering10.3389/fmech.2024.139713110Online publication date: 12-Jul-2024
https://doi.org/10.3389/fmech.2024.1397131
Show More Cited By

Index Terms

Big data and its technical challenges
1. Information systems
  1. Data management systems
    1. Database management system engines
  2. Information systems applications

Recommendations

Challenges for MapReduce in Big Data
SERVICES '14: Proceedings of the 2014 IEEE World Congress on Services

In the Big Data community, MapReduce has been seen as one of the key enabling approaches for meeting continuously increasing demands on computing resources imposed by massive data sets. The reason for this is the high scalability of the MapReduce ...
From Big Data to Big Data Mining: Challenges, Issues, and Opportunities
Proceedings of the 18th International Conference on Database Systems for Advanced Applications - Volume 7827

While "big data" has become a highlighted buzzword since last year, "big data mining", i.e., mining from big data, has almost immediately followed up as an emerging, interrelated research area. This paper provides an overview of big data mining and ...
Hadoop Essentials - Tackling the Challenges of Big Data with Hadoop

Comments

Information & Contributors

Information

Published In

Communications of the ACM Volume 57, Issue 7

July 2014

98 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/2622628

Editor:
Moshe Y. Vardi
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2014

Published in CACM Volume 57, Issue 7

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Popular
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

747
Total Citations
View Citations
27,464
Total Downloads

Downloads (Last 12 months)1,800
Downloads (Last 6 weeks)196

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

AYISI LARBI COSEİ S(2024)Reduction of Losses and Wastage in Seafoods: The Role of Smart Tools and Biosensors Based on Artificial IntelligenceReduction of Losses and Wastage in Seafoods: The Role of Smart Tools and Biosensors Based on Artificial IntelligenceJournal of AI10.61969/jai.1394542Online publication date: 19-Feb-2024
https://doi.org/10.61969/jai.1394542
Zhang LZhu JJin BWei X(2024)Multiview Spatial-Temporal Meta-Learning for Multivariate Time Series ForecastingSensors10.3390/s2414447324:14(4473)Online publication date: 10-Jul-2024
https://doi.org/10.3390/s24144473
Sun YChen CXu YXie SBlum RVenkitasubramaniam P(2024)On the generalization discrepancy of spatiotemporal dynamics-informed graph convolutional networksFrontiers in Mechanical Engineering10.3389/fmech.2024.139713110Online publication date: 12-Jul-2024
https://doi.org/10.3389/fmech.2024.1397131
Erfanian MJagadish HAsudeh A(2024)Chameleon: Foundation Models for Fairness-Aware Multi-Modal Data Augmentation to Enhance Coverage of MinoritiesProceedings of the VLDB Endowment10.14778/3681954.368201417:11(3470-3483)Online publication date: 30-Aug-2024
https://doi.org/10.14778/3681954.3682014
Belal YBen Mokhtar SHaddadi HWang JMashhadi A(2024)Survey of Federated Learning Models for Spatial-Temporal Mobility ApplicationsACM Transactions on Spatial Algorithms and Systems10.1145/366608910:3(1-39)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1145/3666089
Shao WKang YPeng ZXiao XWang LYang YSalim FBaeza-Yates RBonchi F(2024)STEMO: Early Spatio-temporal Forecasting with Multi-Objective Reinforcement LearningProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671922(2618-2627)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671922
Wang XWang PWang BZhang YZhou ZBai LWang Y(2024)Latent Gaussian Processes Based Graph Learning for Urban Traffic PredictionIEEE Transactions on Vehicular Technology10.1109/TVT.2023.330775573:1(282-294)Online publication date: Jan-2024
https://doi.org/10.1109/TVT.2023.3307755
Li MChen SShen YLiu GTsang IZhang Y(2024)Online Multi-Agent Forecasting With Interpretable Collaborative Graph Neural NetworksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.315225135:4(4768-4782)Online publication date: Apr-2024
https://doi.org/10.1109/TNNLS.2022.3152251
Najafi MQadah TSadoghi MJacobsen H(2024)DIBA: A Re-Configurable Stream ProcessorIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.338119236:9(4550-4566)Online publication date: Sep-2024
https://doi.org/10.1109/TKDE.2024.3381192
Huang LYuan JChen SLi X(2024)MDG: A Multi-Task Dynamic Graph Generation Framework for Multivariate Time Series ForecastingIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.33524078:2(1337-1349)Online publication date: Apr-2024
https://doi.org/10.1109/TETCI.2024.3352407
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF Chinese translation

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Challenges for MapReduce in Big Data

From Big Data to Big Data Mining: Challenges, Issues, and Opportunities

Hadoop Essentials - Tackling the Challenges of Big Data with Hadoop

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Digital Edition

Magazine Site

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations