Scaling Data Analytics with Moore's Law

Analyzing the volume, variety and velocity of big data requires the use of modern heterogeneous computing platforms composed of multicores with SIMD execution units, GPUs, clusters, FPGAs and in the future new reconfigurable architectures. However, programming in this environment is extremely challenging due to the need to use multiple low-level programming models and then combine them together in ad-hoc ways. Furthermore, many data analytics algorithms do not take full advantage of modern hardware capabilities. To optimize big data applications both for modern hardware and for modern programmers needs algorithms specialized for modern hardware and a high-level programming model that executes efficiently on heterogeneous parallel hardware. In this talk, I will describe the Delite DSL framework, which uses nested parallel patterns encapsulated in domain specific languages (DSLs). I will describe how a nested parallel pattern based programming model can be used to develop new data analytics algorithms that are optimized for architectures as diverse as multicore/NUMA, clusters, GPUs, FPGAs and a new reconfigurable architecture called Plasticine.