short-paper

HJ-Hadoop: an optimized mapreduce runtime for multi-core systems

Author:

SPLASH '13: Proceedings of the 2013 companion publication for conference on Systems, programming, & applications: software for humanity

Pages 111 - 112

https://doi.org/10.1145/2508075.2514875

Published: 26 October 2013 Publication History

Get Access

Abstract

We introduces HabaneroJava-Hadoop, an extension to the HadoopMapReduce system that is optimized for multi-core machines. HJ-Hadoop exploits intra-JVM parallelism that increases memory efficiency of each node. Results show a significant improvement in the amount of data each MapReduce job could process and load balance across cores for certain applications.

References

[1]

Hadoop. http://hadoop.apache.org. URL http://hadoop.apache.org.

Google Scholar

[2]

V. Cavé, J. Zhao, J. Shirako, and V. Sarkar. Habanero-java: the new adventures of old x10. PPPJ '11.

Google Scholar

Cited By

View all

Finnerty PKamada TOhta C(2023)Automatically balancing relocatable distributed collectionsConcurrency and Computation: Practice and Experience10.1002/cpe.771735:27Online publication date: 23-Apr-2023
https://doi.org/10.1002/cpe.7717
Finnerty PKawanishi YKamada TOhta C(2022)Supercharging the APGAS Programming Model with Relocatable Distributed CollectionsScientific Programming10.1155/2022/50924222022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/5092422
Finnerty PKamada TOhta C(2022)Integrating a global load balancer to an APGAS distributed collections libraryProceedings of the Thirteenth International Workshop on Programming Models and Applications for Multicores and Manycores10.1145/3528425.3529102(55-64)Online publication date: 2-Apr-2022
https://dl.acm.org/doi/10.1145/3528425.3529102
Show More Cited By

Index Terms

HJ-Hadoop: an optimized mapreduce runtime for multi-core systems
1. Software and its engineering
  1. Software notations and tools
    1. Compilers
      1. Runtime environments

Recommendations

Hybridhadoop: CPU-GPU hybrid scheduling in hadoop
Abstract
As a GPU has become an essential component in high performance computing, it has been attempted by many works to leverage GPU computing in Hadoop. However, few works considered to fully utilize the GPU in Hadoop and only a few works studied ...
HybridHadoop: CPU-GPU Hybrid Scheduling in Hadoop
HPCAsia '21: The International Conference on High Performance Computing in Asia-Pacific Region

As a GPU has become an essential component in high performance computing, it has been attempted by many works to leverage GPU computing in Hadoop. However, few works considered to fully utilize the GPU in Hadoop and only a few works studied utilizing ...
Performance comparison between Hadoop and HAMR under laboratory environment

With the development of the Internet technology, data explosion is about to take place. To handle such enormous amount of data, including storing, organizing and analyzing, the capability of a single machine is far from sufficient. Therefore, it is ...

Comments

Information & Contributors

Information

Published In

SPLASH '13: Proceedings of the 2013 companion publication for conference on Systems, programming, & applications: software for humanity

October 2013

192 pages

ISBN:9781450319959

DOI:10.1145/2508075

Co-chair:
Antony Hosking
Purdue University, USA
,
General Chair:
Patrick Eugster
Purdue University, USA

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2013

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

SPLASH '13

Sponsor:

SIGPLAN

SPLASH '13: Conference on Systems, Programming, and Applications: Software for Humanity

October 26 - 31, 2013

Indiana, Indianapolis, USA

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
249
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Finnerty PKamada TOhta C(2023)Automatically balancing relocatable distributed collectionsConcurrency and Computation: Practice and Experience10.1002/cpe.771735:27Online publication date: 23-Apr-2023
https://doi.org/10.1002/cpe.7717
Finnerty PKawanishi YKamada TOhta C(2022)Supercharging the APGAS Programming Model with Relocatable Distributed CollectionsScientific Programming10.1155/2022/50924222022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/5092422
Finnerty PKamada TOhta C(2022)Integrating a global load balancer to an APGAS distributed collections libraryProceedings of the Thirteenth International Workshop on Programming Models and Applications for Multicores and Manycores10.1145/3528425.3529102(55-64)Online publication date: 2-Apr-2022
https://dl.acm.org/doi/10.1145/3528425.3529102
Zhang HHuang HWang L(2017)MRapid: An Efficient Short Job Optimizer on Hadoop2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS.2017.100(459-468)Online publication date: May-2017
https://doi.org/10.1109/IPDPS.2017.100
Paulino HDelgado N(2017)Cache-conscious run-time decomposition of data parallel computationsThe Journal of Supercomputing10.1007/s11227-016-1721-z73:8(3366-3390)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1007/s11227-016-1721-z
Magro DPaulino H(2016)In-cache MapReduce: Leverage Tiling to Boost Temporal Locality-Sensitive MapReduce Computations2016 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER.2016.33(374-383)Online publication date: Sep-2016
https://doi.org/10.1109/CLUSTER.2016.33

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Hybridhadoop: CPU-GPU hybrid scheduling in hadoop

HybridHadoop: CPU-GPU Hybrid Scheduling in Hadoop

Performance comparison between Hadoop and HAMR under laboratory environment