Heterogeneous Big Data Parallel Computing Optimization Model using MPI/OpenMP Hybrid and Sensor Networks

EXPRESSION OF CONCERN: ACM is issuing a formal Expression of Concern for all papers published in the TOSN Special Issue on Green Communications and Sensor Networks with Machine Intelligence for Smart Cities while a thorough investigation takes place with regards to the integrity of the peer review process. ACM strongly suggests that papers in this special issue not be cited in the literature until ACM's investigation has concluded and final decisions have been made regarding the integrity of the peer review process.

For the heterogeneous big data parallel computing model, two levels of parallelism between nodes are not considered, resulting in low efficiency of heterogeneous big data parallel computing and bandwidth to send and receive information, high communication overhead, long model running time and small computational volume. In the paper, we propose an optimization model of heterogeneous big data parallel computing based on a hybrid Multi Point Interface (MPI)/Open Multi-Processing (OpenMP) and Sensor Networks. First, the processor characteristics of heterogeneous big data architecture is analyzed, the parallel tasks among processors are divided, collect the heterogeneous big data to be computed and cluster them, and use the processing results as the input items of the model. Then, a parallel load balancing mechanism is established to optimally divide the parallel computing load of heterogeneous big data, and a parallel computing optimization program is written by combining the hybrid programming mode of MPI and OpenMP and using the hybrid MPI/OpenMP, and finally, the parallel computing optimization of heterogeneous big data is realized by optimizing the parallel communication and determining the model parameters. The results show that the proposed model has a communication bandwidth of 510Mbps, a computational volume of 1.16GB, a model runtime of 24s, and an improved network bandwidth utilization of 93%, which can effectively reduce the communication overhead, and improve the efficiency of parallel computing and bandwidth sending and receiving information in sensor networks, and shorten the model running time.