A Hybrid Parallel Computing Model to Support Scalable Processing of Big Oceanographic Spatial Data

Abstract

Oceanographic sciences are facing big challenges due to the deluge of big data. As of 2010, the amount of new data stored in the world main countries, led by the US, has grown over 7 exabytes. Although the computer hardware is quickly evolving, with faster processor frequency, multi-core technology, and larger memory, traditional reprocessing paradigm on a single-desktop basis still suffers from significant limitations in its low computational efficiency and scalability. In this paper, we report our effort in developing a hybrid parallel computing model which utilizes Graphic Processing Unit (GPU) to accelerate Hadoop Map Reduce system. In each computing node, the actual reprocessing is offloaded from a CPU to a GPU to further boost up the system performance. We describe the architecture design of the proposed model and the automated task/data assignment on each GPU-enabled compute node. Electronic Navigational Charts in ocean fields involves a huge amount of spatio-temporal data. Reprojection of these data between different coordinate reference systems, which is a computation-intensive task, is selected as the use case. Systematic experiments were conducted to demonstrate the good performance of the proposed model.