GPUMap: A Transparently GPU-accelerated Python Map

Function
Ivan Pachev Chris Lupo
California Polytechnic State University California Polytechnic State University
San Luis Obispo, California, USA San Luis Obispo, California, USA
ipachev@calpoly.edu clupo@calpoly.edu

ACM Reference Format: Building GPU-accelerated distributed applications is possible

Ivan Pachev and Chris Lupo. 2017. GPUMap: A Transparently GPU-accelerated using CUDA bindings such as PyCUDA [3] and JCUDA [15]. How-
Python Map Function. In PyHPC’17: PyHPC’17: 7th Workshop on Python for ever, these libraries provide almost no simplifcations to GPU pro-
High-Performance and Scientifc Computing, November 12–17, 2017, Den- gramming, requiring programmers to already be familiar with GPU
ver, CO, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/ programming prior to using them. Furthermore, requiring program-
3149869.3149875
mers to explicitly incorporate traditional GPU programming into
their distributed applications conficts with the purpose of the dis-
tributed computing frameworks, which is to abstract the underlying
1 INTRODUCTION
distributed system.
GPU computing has been adopted by many programmers to solve There are also software packages that provide simplifed GPU
a wide range of problems with high performance. Creating a GPU- acceleration to distributed computing frameworks such as MapRe-
accelerated application or refactoring an existing application to be duce [1] and Spark [17]. Although these packages provide a sim-
GPU-accelerated is not always a trivial task. GPU programmers plifed GPU programming experience, they do not do so in a com-
are often required to be familiar with the architecture of the GPU pletely abstracted manner.
in order to create an efcient GPU program. For programmers The contributions of this paper are two-fold. The frst contri-
without any understanding of GPU architecture, learning GPU bution is GPUMap, a GPU-accelerated map function for Python.
programming in order to incorporate GPU-acceleration into their GPUMap hides all the details of the GPU from the programmer, and
applications is time consuming and sometimes difcult. allows the programmer to accelerate programs written in normal
There are some existing eforts to develop software packages Python code that operate on arbitrarily nested objects made up of
that attempt to simplify GPU programming in higher-level lan- primitive data using a majority of Python syntax. Using GPUMap,
guages such as Java and Python. They require programmers to certain types of Python programs are able to be accelerated up to
be familiar with the traditional GPU programming model which 100 times over normal Python code.
involves some understanding of GPU threads, memory, and kernels. The second contribution is GPU-accelerated RDD (GPURDD),
These software packages usually restrict the use of object-oriented which is a Resilient Distributed Dataset (RDD) that is used with
programming. As a result, prior to using these software packages, Spark. GPURDD incorporates GPUMap into its map, filter, and
programmers are required to transform the data they would like to foreach methods in order to allow Spark applications to make use
operate on into arrays of primitive data. of the simplifed GPU acceleration provided by GPUMap. These
Just as individual systems with GPU-computing capability have transformations can be used with normal Python functions and do
become more available, so too have high-performance distributed not require objects in the RDD to be restructured in order to use
systems. A limiting factor in the computing power of many modern them.
cluster computing frameworks is that, although they are scalable, This paper is organized as follows. Section 2 discusses back-
the frameworks are typically constrained to CPU-exclusive dis- ground information on GPU computing, Python, and Spark. Sec-
tributed computing applications. This means that these frameworks tion 3 discusses related works that attempt to provide simplifed
may lack the ability to harness the powerful GPUs belonging to GPU acceleration in both individual and cluster computing environ-
each system. Compute intensive tasks such as machine learning, ments. Section 4 explains the design and implementation of both
image processing, and data analytics will beneft from distributed GPUMap and GPURDD. Section 5 analyzes performance bench-
computation on systems with GPUs. marks that use GPUMap and GPURDD, and Section 6 concludes
and provides directions for future research.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for proft or commercial advantage and that copies bear this notice and the full citation
on the frst page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or 2 BACKGROUND
republish, to post on servers or to redistribute to lists, requires prior specifc permission
and/or a fee. Request permissions from permissions@acm.org. This section discusses necessary background information related
PyHPC’17, November 12–17, 2017, Denver, CO, USA to GPUMap, the implementation of GPUMap itself, and testing
© 2017 Copyright held by the owner/author(s). Publication rights licensed to Associa- GPUMap. Some basic background information on GPU computing,
tion for Computing Machinery.
ACM ISBN 978-1-4503-5124-9/17/11. . . $15.00
Python, and Spark is also presented.
https://doi.org/10.1145/3149869.3149875
2.1 GPU Computing 2.2.1 Object Representation. Python objects are stored in a hash
GPUs can be used as a powerful parallel processor that is capable of table called a dictionary [10]. The keys to the dictionary are the
more foating point operations and has higher streaming memory names of the felds or methods of an object. The values of the
bandwith than even the highest-end CPUs [7]. Although GPUs can dictionary are the objects or methods referred to by the keys.
be used as general purpose parallel processors, GPU programming When a feld or method is accessed in Python, the name of the
usually focuses on quickly performing batches of numerical com- feld or method is looked up in the object’s dictionary at runtime.
putations, rather than running full multithreaded, object-oriented If the name belongs to a feld, the object contained in the feld is
programs. Developers must take into account the limitations of the returned. All functions and methods are callable objects in Python,
GPU when writing optimized GPU programs. so when the name of a method is looked up in an object’s dictionary,
The following is a discussion of some CUDA GPU-programming the callable object representing the method is returned. This callable
basics, as they are necessary in understanding the implementation object can either be called or passed around as a normal object.
details of GPUMap. 2.2.2 Map Function. One of the functional programming com-
2.1.1 Thread Hierarchy. Threads are organized into a thread hi- ponents provided with Python is the map function. The map function
erarchy [6] consisting of blocks and grids. Both of these hierarchical provides an efective abstraction for transforming a list by applying
levels can be indexed in one, two, or three-dimensions. Each thread a function to each element by accepting a function f and a list L,
operates on a piece of data based on its position in the block and and applying f to each element in L to produce a new list.
the position of the block in the grid. In Python 3, the value returned by the map function is a generator.
A generator is like an iterator, but the elements returned by the
2.1.2 Memory Hierarchy. There are three diferent types of generator are lazily produced when the generator is iterated upon.
memory on the GPU that are used for GPU computing: thread-local Thus, when the map generator is iterated upon, f is applied to each
memory, shared memory, and global memory [6]. Thread-local successive element over L, returning the result.
memory is available for each thread and is used for the individual
2.2.3 Closures. Python closures are supported, allowing func-
runtime stack of each thread. This memory stores local variables
tions to refer to variables from an outer scope. Functions are objects
and information about function calls. Shared memory is larger than
that can be stored and called later, so function objects must store
thread-local memory and is shared between threads in a block. The
references to these outer scope variables for later access when the
largest and slowest type of memory is global memory. Global mem-
function object is called.
ory can be accessed by any threads in any block and is the only
Python stores these references by creating a mapping between
GPU memory that is directly accessible to the host processor.
the variable names and the objects they refer to in the form of a
2.1.3 Serialization. Prior to running any GPU code, the data dictionary. This dictionary can be accessed by using the __clos__
that will be operated on must be serialized into a stream of primi- feld of a function object.
tives. Space for this serialized data must be allocated in the GPU’s
global memory. Serialized data must be copied from the host to the 2.3 Apache Spark
allocated memory block on the GPU. Apache Spark is an open-source, cluster-computing framework that
Once the execution on the GPU is complete, the data needs to provides abstractions for the underlying distributed system that
be copied back to the host. Once the data is copied back to the host, allow programmers to harness the power of the system without
the data must be deserialized and inserted into its originating data needing to understand how the system works [17].
structures. Spark is efcient for iterative and interactive tasks due to the
nature of its underlying data model, the resilient distributed dataset
2.1.4 Kernels. The program to be run on the GPU is written in (RDD). A RDD is a read-only, fault-tolerant collection of items that
the form of a kernel [6]. The kernel is a function that is executed in is partitioned across a cluster and can be operated on in parallel [16].
parallel for each thread. A pointer to the data set to be operated on Spark’s workers are long-lived processes that keep RDD partitions
by the GPU is passed to the kernel function. persistent in memory between operations, allowing them to be
easily reused in future operations [16].
2.2 Python Because RDDs are typically operated on in parallel, users should
Python is a general purpose, interpreted programming language see performance increases provided by GPU-accelerated implemen-
that supports object-oriented programming, functional program- tations of RDD operations on the worker nodes.
ming, and procedural programming. The language is dynamically
typed, meaning that functions can be passed any types of argu- 3 RELATED WORK
ments and can return any type. In addition, felds and variables can There are other projects that aim to provide simplifed GPU pro-
store data of any type. Python does this by performing late binding, gramming to languages such as Java and Python, including Root-
which means that variable names are bound to their corresponding beer, Aparapi, and Numba. These projects allow programmers to
objects at runtime. implement GPU algorithms and produce GPU-accelerated appli-
The following is a discussion of object structure in Python, cations without needing to write any CUDA or OpenCL code. In
Python’s map operation, and Python closures as they are necessary addition, there are some projects that attempt to provide GPU-
in understanding the implementation details of GPUMap. acceleration in both MapReduce and Spark, such as Multi-GPU
2
MapReduce, Hadoop+Aparapi, and Spark-Ucores. This section dis- 3.5 Hadoop+Aparapi
cusses these related projects. Hadoop+Aparapi provides an improved interface over traditional
GPU programming [5]. The developers were not able to completely
3.1 Rootbeer abstract the GPU component due to limitations in Aparapi when
Hadoop+Aparapi was created. Thus, simplifcation of the Hadoop+Aparapi
Rootbeer is a Java framework that aims to automate GPU program- API was not possible. The Aparapi team has added features that
ming steps such as serialization of the input data, creating and may help simplify the Hadoop+Aparapi’s API such as the usage of
launching GPU kernels, and deserialization of the data back into objects and lambda expressions in the GPU kernel code [2].
the CPU memory. Rootbeer allows developers to write their parallel
code in Java by implementing the Kernel interface which contains 3.6 Spark-Ucores
only one method: gpuMethod. Rootbeer automatically fnds all felds
that are reachable from the Kernel class and serializes them to the Spark-Ucores also uses Aparapi to translate Java code into OpenCL[12].
GPU [9]. The Spark-Ucores team has also forked Aparapi in order to pro-
Rootbeer allows usage of almost all Java features, including ar- vide support for FPGAs and APUs, resulting in the creation of
rays, composite objects, dynamic memory allocation, strings, and Aparapi-Ucores [12]. Spark-Ucores provides GPU-accelerated im-
exceptions. However, in order to implement a more complex appli- plementations of a few parallel operations on RDDs, including map,
cation, some of the underlying CUDA functionality, such as thread mapPartitions, and reduce.
confguration, shared memory and CUDA thread synchronization Spark-Ucores does not provide an abstraction for its GPU compo-
must be manually specifed [9]. nents, so programmers must have at least some GPU experience. In
order to use Spark-Ucores, the user must restructure their existing
Spark code, which may further discourage usage of Spark-Ucores.
3.2 Aparapi
Aparapi is another Java library that attempts to automatically GPU- 4 IMPLEMENTATION
accelerate user-supplied Java code [2]. However, rather than trans- GPUMap is open-source software, and may be downloaded from
lating the Java bytecode into CUDA, the bytecode is translated into the repository at: https://github.com/ipachev/py_gpumap.
OpenCL. The primary goal of GPUMap is to provide transparent GPU-
Aparapi only supports usage of primitives and allows operation acceleration, which involves automatic serialization, code trans-
on them using normal Java syntax, as well as many of the functions lation, execution of the translated code, and deserialization. The
provided by java.lang.Math [2]. Aparapi does not support objects, programmer should be able to write normal Python code that pro-
static methods, recursion, overloaded methods, exceptions, dynamic vides a function f and a list L and call gpumap(f, L) to produce a
allocation, synchronization, or switch statements [2]. list L ′ , the same way that they would normally call map(f, L).

4.1 Requirements
3.3 Numba
The implementation of GPUMap imposes the following require-
Numba is a just-in-time (JIT) compiler for Python targeted towards ments to support GPU execution.
scientifc computing. Numba provides limited GPU-acceleration,
allowing programmers to operate on NumPy arrays in parallel • Objects must contain only integers, foating point numbers,
using the GPU [4]. GPU-acceleration with Numba is much easier booleans, or other objects.
than writing CUDA code because Numba provides considerable • Objects of the same class must have the same felds and
simplifcations to the traditional GPU programming model. the same types in their corresponding felds, i.e. must be
Numba does not require programmers to be well-versed in CUDA homogeneous.
programming. Instead, the code that is to be translated can be • Objects cannot contain members of the their own class either
written in a subset of Python. The reason only a subset of Python directly or indirectly.
is allowed is because Numba must be to be able to infer all types • Lists must contain only one class of objects.
in order to generate the proper GPU code [4]. This means that • Functions or methods must be passed arguments of the same
object-oriented code cannot be used. type every time they are called.
• Functions or methods must return the same type every time
they are called.
3.4 Multi-GPU MapReduce (GPMR) • When a function is called, the function must call the same
Multi-GPU MapReduce is a fexible MapReduce implementation functions or methods every time.
that works on clusters where the compute nodes have GPUs. Multi-
GPU MapReduce outperforms the normal CPU implementation 4.2 Invocation
of MapReduce, as well as other single-GPU implementations of When the programmer calls gpumap(f, L), the following steps are
MapReduce [14]. Although the framework performs well and is taken in order to perform the desired map operation:
fexible, Multi-GPU MapReduce does not provide a high-level ab- (1) f is applied to the frst element of L, L 0 , to produce L 0′ and
straction for the underlying system, which may cause difculty for runtime inspection is performed to analyze every function
users without GPU programming experience. call.
3
 (2) The felds of L 0 and L 0′ are inspected to collect data about 4.5 Code Generation
the classes of L 0 and L 0′ . In order to operate on a list L by applying a function f to each
(3) If f is a closure, any objects that are included in the clo- element in the list on the GPU, the necessary CUDA C++ class
sure bindings are also inspected and information is collected defnitions and function/method defnitions must be generated.
about their classes. This process is non-trivial, and requires the emulation of Python’s
(4) CUDA C++ class defnitions are created for the necessary pass-by-reference behavior by passing all objects as references to
classes by using the information collected during runtime functions.
inspection and object inspection. Classes, methods, and functions are generated from the data
(5) Any functions and methods, including constructors, that are structures extracted from runtime inspection.
called when applying f to L 0 are translated into CUDA C++.
(6) All of the elements of L 1...n are serialized to the GPU. Any Built-in Functions. Support was added for built-in functions such
of the objects or lists that have closure bindings in f are also as math functions, len, print, and others. Some of these built-in
serialized. functions have existing counterparts in CUDA C++, such as the
(7) The map kernel, which includes all class, function, and method math functions. Other functions that do not have existing counter-
translations, is compiled and executed on the GPU, applying parts, such as len, are implemented in C++ and are supplied in a
the translation of f , f ′ , to each element in the serialized header during compilation. Due to the fact that the names of built-
version of L 1...n . in Python functions do not always match up with built-in CUDA
(8) The serialized input list, L 1...n , and any closure objects or C++ functions, translating the names of built-in Python functions
lists are deserialized and the data is re-incorporated into the may be necessary.
original objects.
(9) The output list L 1...n
′ is deserialized and is used to populate 4.6 Kernel Generation
a list of objects based on the structure of L 0′ . The fnal step of code generation is fnalizing the CUDA kernel
(10) L 0′ is prepended to L 1...n
′ to form L ′ as desired and L ′ is function that will be executed on the GPU. The kernel function is a
returned. function that is executed by each GPU thread. In order to parallelize
the map process, the GPU thread will apply the top-level function
4.3 Implementation Details to a diferent list item.
There are many details and important algorithms used in the im-
plementation of GPUMap. Some of the most pertinent steps are 4.7 Serialization
described here, but the full discussion of the implementation is be- When calling gpumap(f, L) with a function f and a list L, L and
yond the scope of this paper. For full details of the implementation, any closure variables of f must be serialized and copied to the GPU.
the reader is referred to Reference [8], and the source repository The list L and any closure variables are not cached on the GPU and
on GitHub. must be serialized for every call to gpumap, although this may be
addressed by future work to improve performance.
4.4 Runtime Inspection After the translated code is executed, L and f ’s closure variables
Prior to doing any code translation or serialization, some data must must be copied back to the host and deserialized.
be collected about the functions and methods that will need to 4.7.1 Serializing Objects. Prior to copying an object to the GPU,
be called, as well as the objects that will need to be operated on. the object must be serialized into the proper format so that the
This data is acquired through inspection of the felds of objects and object can be processed by the translated CUDA code. Serializing
tracing function execution when the given function is applied to a Python object involves collecting all of its non-contiguous parts
the frst element in the list. Runtime inspection consists of call and and collecting them in a contiguous section of memory as normal
object inspection, which are performed to extract representations binary data, as depicted in Figure 2.
of functions, methods, and objects. In order to collect all the data in an object, including the data in
Several pieces of information need to be known about a class of its nested objects, the object’s felds can be recursively examined.
objects in order to generate the appropriate CUDA class defnitions The order in which the felds are accessed must be in the same order
and properly serialize objects of that class. This information is as the class defnition that is created during the class generation
stored in a Class Representation. phase.
The Class Representation is a recursive data structure that may
contain multiple other Class Representations for each feld. A Class 4.7.2 Serializing Lists of Objects. The process for serializing an
Representation is extracted by examining all of the felds of a sample entire list is similar to serializing a single object so that objects,
object, obj, by iterating through obj’s felds as a normal Python dict, whether or not they originate from a list, can be accessed and
using obj.__dict__. This dict contains a mapping from obj’s feld manipulated the same way, using the same C++ class defnition. The
names to the objects contained in those felds. For each entry in same is true for deserialization. However, in the case of deserializing
this dict, the feld name is recorded, and a Class Representation is the output list of the map operation, the objects do not yet exist
extracted and recorded for the object contained in the feld. Figure 1 in the Python code, so a slightly diferent approach must be taken
depicts a sample extraction of a Class Representation of an object that also involves object creation, so the data can be unpacked into
of a class called classA. objects.
4
 Figure 1: Sample Extraction of a Class Representation

Figure 2: A normal Python object and its serialized counterpart

For each object to be instantiated in the output list, the frst a partition. The mapPartitions method accepts a function that
output object, which was created during runtime inspection, is takes an iterator of the input type and produces an iterator of the
deep copied and is used as a skeleton for the new object. transformed type, making д an acceptable candidate.
The object is then inserted into the output list. Once the correct The closure д that is implemented inside the body of map returns
number of objects have been created, populated, and inserted into a map generator that applies the function passed to map, f , to each
the output list, the output list is returned. element returned by the iterator supplied to д. Map generators
are created by using Python’s built-in map function. Each time the
4.8 Integration in Spark map generator is iterated upon, the generator lazily applies f to a
new element from the iterator passed to д and produces the output.
GPUMap can be integrated in a variety of transformations and
In order to obtain an iterator to an entire partition, д is passed
actions that can be performed on Spark RDDs. This section describes
to mapPartitions, where д will be given the partition iterator.
the implementation of GPURDD, which is a class that extends RDD
The call to mapPartitions returns a handle to an RDD that will
and provides alternative implementations of map, flter, and foreach.
eventually contain the items transformed using д, once the RDD is
The restrictions described at the beginning of this section regarding
evaluated, and this is returned from the RDD’s map method.
lists also extend to Spark RDDs.
In order to incorporate GPUMap into GPURDD’s map function,
4.8.1 Map. The map method on a Spark RDD allows the pro- the implementation of д must be altered to create д ′ . The function
grammer to perform a transformation on the RDD by individually д ′ must still take an iterator of f ’s input type and return an iterator
applying a function f to each element of the RDD to produce a new of f ’s output type. The application of f on many items from the
RDD containing transformed elements. iterator must be evaluated immediately in parallel, rather than
The existing implementation of RDD’s map method defnes a producing a map generator to evaluate the application of f to each
function д that takes an iterator of the input type of f and returns element from an iterator in sequence. This means that access to all
an iterator of the output type of f . Because f is stored in the closure the elements in a partition simultaneously is necessary, which can
bindings of д, f can be passed along with д. This closure д is then be achieved by exhausting the iterator into a list. This list can then
passed to mapPartitions so that f is applied to each element in be passed into GPUMap, along with f , in order to apply f in parallel
5
and produce a transformed list. Because GPUMap outputs a list iterator over a partition, so that the elements of the partition can be
and not an iterator, д ′ must return an iterator over the list, rather fltered. An RDD handle is returned by the call to mapPartitions.
than just the list itself. The last step of GPURDD’s map method is The purpose of incorporating GPUMap into GPURDD’s filter
to return the return value of the call to mapPartitions, which is a method is to attempt to speed up the evaluation of f on each
handle to an RDD that will eventually contain the values that will element of the partition. Due to the fact that when using GPUMap,
have been transformed using д ′ . the input list and output list must have a one-to-one correspondence,
Once an action is performed on the resulting GPURDD in order to GPUMap cannot be directly used to flter the elements. However, the
evaluate all of the transformations, д ′ will be called with an iterator results of applying f to each item can be computed using GPUMap
to the partition elements in order to transform them. Although and can be subsequently used to remove elements.
the partition iterator is exhausted by д ′ , Spark’s lazy evaluation In order to implement GPURDD’s filter method, a function
model is still preserved because д ′ is passed to mapPartitions. д ′ must be created to be used with mapPartitions, similar to
The mapPartitions method will only call д ′ when the time comes GPURDD’s implementation of map. First, the iterator passed to
to evaluate the RDD. RDDs are evaluated when an action, such as д ′ must be exhausted to produce a list of items that can be operated
collect, count, or foreach is performed on them. on in parallel. Then, gpumap is called with f and the list of items
to produce a list of boolean values indicating whether or not to
4.8.2 Foreach. The foreach method on a Spark RDD allows
keep an entry in the list of items. Once the list of items and the
the programmer to apply a function f to each element of an RDD
list of booleans are available, Python’s zip iterator can be used to
without transforming the RDD. Instead, the foreach method is
provide tuples consisting of the item itself and its corresponding
used to produce side-efects in the elements of the RDD.
boolean. Then, Python’s built-in filter function is used to create a
The existing implementation of foreach makes use of the RDD’s
flter iterator from the zip iterator. This flter iterator will not return
mapPartitions method, similarly to the way map method does. The
tuples where the second feld of the tuple, the boolean, is false. Then
foreach method simply defnes a function д that iterates through
the tuples returned by the flter iterator can be converted back into
a partition iterator, applying f to each item of the partition, ig-
items by using a map generator. A map generator is created by
noring the return value of the call to f . In order to comply with
using Python’s built-in map function that maps a tuple yielded by
the fact that any function passed to mapPartitions must return
the flter iterator to the tuple’s frst feld, which is the item itself.
an iterator, д returns an empty iterator. This function, д, is passed
This map generator serves as an iterator over the fltered items and
to mapPartitions, which creates a handle to an empty, dummy
is returned by д ′ . This process is illustrated in Figure 3.
RDD. The reason an empty RDD is created is because д returns an
Then д ′ is passed to mapPartitions and the resulting RDD
empty iterator. When this dummy RDD is evaluated, f is applied
handle is returned. Once an action is performed on the resulting
to each element of the source RDD. In order to force evaluation of
RDD, then the RDD will be evaluated and д ′ will be called on an
the dummy RDD, the foreach method calls the count method of
iterator over each partition, as with GPURDD’s implementation of
this dummy RDD. The handle to the dummy RDD is not returned
map.
from foreach, as a handle to the dummy RDD is not useful.
GPUMap preserves side-efects, and can be efectively incorpo-
rated into foreach. The approach taken to incorporate GPUMap is
5 EXPERIMENTS
similar to the approach taken with GPURDD’s map method. A func- In order to determine the types of workloads that can be accelerated
tion д ′ is defned that takes an iterator over a partition and returns using GPUMap, performance benchmarks are performed. The tests
an empty iterator. The body of д ′ simply consists of exhausting the used for performance benchmarking GPUMap are the n-body test
partition iterator into a list, and calling gpumap with f and the list and the bubble sort test. The tests used for performance bench-
created by exhausting the iterator. The return value of the call to marking GPURDD are the shell sort test and the pi estimation tests.
gpumap can be discarded as it is not useful. These benchmarks use a variety of algorithms with diferent time
Then, д ′ is passed to mapPartitions to create a handle to a complexities, allowing us to examine the viability of GPUMap or
dummy RDD and, similarly to RDD’s foreach method, evaluation GPURDD in these diferent scenarios.
of the dummy RDD is forced using the handle’s count method in The experimental setup consists of machines ftted with:
order to apply д ′ to each partition. • Intel Xeon E5-2695 v3 CPU @ 2.30GHz
4.8.3 Filter. The filter method on a Spark RDD allows the • NVIDIA GeForce GTX 980
programmer to transform an RDD by removing elements from • 32 GB Memory
the RDD by applying a function f to each element that returns a • CentOS 7
boolean indicating whether or not to keep the element. The NVIDIA GeForce GTX 980 has 4GB of memory and 2048
The filter method is implemented very similarly to how map CUDA cores running at 1126 MHz. The GTX 980 supports CUDA
is implemented, incorporating the use of mapPartitions. This Compute Capability 5.2 which allows up to 1024 threads per block
method defnes a closure д that takes an iterator that provides and a maximum one-dimensional grid size of 231 − 1. Although the
elements of the RDD and returns an iterator that provides elements maximum number of threads per block is 1024, the benchmarks all
that did not get fltered. The closure д calls Python’s built-in filter use a block size of 512 × 1 × 1. The grid size is ⌈n/512⌉ × 1 × 1 where
function with f to create an iterator that produces items from an n is the size of the input list. This confguration allows GPUMap to
iterable for which a f returns true and simply returns this iterator. achieve an occupancy of 100%, meaning that all 2048 CUDA cores
Then, д is passed to mapPartitions, which provides д with an on the GTX 980 are able to be used concurrently.
6
 Figure 3: GPURDD flter method

Each machine shares the same GPU to run a graphical user

interface as well as CUDA kernels. As a result, the maximum possi-
ble CUDA kernel runtime is constrained, which in turn limits the
maximum possible data sizes that can be used.
In the cases where just GPUMap is benchmarked, only a single
machine is used. In the cases where GPURDD is benchmarked, a
Spark cluster consisting of 10 of these machines is used. The Spark
cluster is confgured so that each worker node only has one worker
process and the worker process is only allowed to use one core. This
confguration is used because there is only one GPU per machine
and so executing GPUMap simultaneously from diferent processes
is not possible.
The remainder of this section will discuss the diferent bench-
marks and perform a performance evaluation. Overall, the results
demonstrate that both GPUMap and GPURDD are not viable for
O(n) algorithms but provide considerable performance improve-
ments to algorithms with larger time complexities.
Figure 4: Speed-up using GPUMap with the N-body bench-
5.1 N-body Benchmark mark
The n-body test is an all-pairs n-body implementation based on the
outer loop approach from [13]. This test is used as a performance
• Code generation, where the appropriate CUDA code is gen-
benchmark for GPUMap. This is an O(n2 ) algorithm due to the fact
erated and compiled using the information acquired from
that on each step of the simulation, for each body, each other body
frst call, as well as AST inspection.
must be considered.
• Serialize, where the closure variables as well as remaining
Prior to running the simulation, a warmup on 256 bodies is
objects in the list are inspected, serialized, and copied to the
performed using both Python’s map and GPUMap. Then, the simu-
GPU.
lation is run starting with two bodies, all the way up to 8192 bodies
• Run, where the CUDA kernel is executed in order to operate
by powers of two. For each number of bodies, the simulation is run
on the data in parallel.
fve times and the average execution time is computed.
• Deserialize, where the serialized data is copied back from
Figure 4 shows the speed-up achieved by using GPUMap over
the GPU and unpacked into its originating objects.
normal Python running the exact same code. With less than 256 bod-
ies, there is no speed-up and GPUMap is not viable. However, start- As expected, due to the fact this algorithm is O(n 2 ), the frst call
ing with 1024 bodies, there is a considerable speed-up of slightly and run stages’ durations increase much more quickly than the
above 12 times. With 8192 bodies, the program is able to execute serialization and deserialization stages’ durations with increases
about 249 times faster. in body count. Serialization and deserialization durations increase
The reason for this increase in performance is that because this with increases in body count, but not as quickly as frst call and run
all-pairs n-body simulation is an O(n 2 ) algorithm, the amount of durations due to the fact that serialization and deserialization are
work needed to be done increases faster than the data that needs to only O(n). In addition, code generation seems relatively constant
be serialized for increasing body count. Thus, for larger numbers of across changes in input data size because the code that is generated
bodies, the processing duration is not outweighed by serialization. is independent of the input data size.
Figure 5 shows a breakdown of the run times of diferent stages Overall, GPUMap is able to produce a considerable performance
of GPUMap for the diferent input data sizes. These stages are: improvement of 249 times over the exact same n-body simulation
code running through Python on the largest tested data set. How-
• First call, where runtime inspection is performed by applying ever, GPUMap is not useful for n-body simulations with very small
the given function to the frst item in the list. data sets as GPUMap actually causes a slowdown.
7
Figure 5: Stage duration using GPUMap with the N-body Figure 6: Speed-up using GPUMap to sort 1000 lists using
benchmark bubble sort

5.2 Bubble Sort Benchmark

The bubble sort test is a simple test that involves sorting multiple
lists of randomly generated integers in parallel using bubble sort,
an O(n2 ) sorting algorithm. This test is also used as a performance
benchmark for GPUMap.
When sorting multiple lists in parallel using the GPU, some lists
may be more poorly sorted than others, causing certain threads
take longer to complete. This means that the execution time of all
the threads on the same block is always as long as the time taken
to sort the most poorly sorted list, resulting in more consistent
worst-case scenario performance for bubble sort.
Prior to performing the benchmark, a warmup is performed by
using both Python’s map and GPUMap to sort 1000 lists of 256
elements each. The benchmark consists of sorting 1000 lists of
particular length, starting with lists of size 2 all the way to lists
of size 4096 using both Python’s map as well as GPUMap. Then
another benchmark is performed that sorts 5000 lists of length from Figure 7: Stage duration using GPUMap to sort 1000 lists us-
2 to 4096 by powers of 2. For each data size, the benchmark is run ing bubble sort
5 times and an average runtime is computed.
Figure 6 shows the performance increase that can be obtained
from using GPUMap to sort 1000 lists in parallel with bubble sort this is an O(n2 ) algorithm, the durations of the frst call and run
over normal Python running the exact same code, sorting the same stages increase much quicker than the durations of the serialize
lists. For lists of size less than 32, GPUMap is not viable as the serial- and deserialize stages with increases in input data size. As previ-
ization duration outweighs the kernel runtime duration. For lists of ously mentioned, serialization and deserialization is O(n), so the
size 32 or greater, GPUMap is able to outperform normal Python’s durations of these stages do not increase as quickly with increases
map function. The largest speed-up, 33.5 times, is obtained for the in input data size. Furthermore, the duration of code generation is
lists of size 4096. However, this speed-up is not much greater than independent of the input data size as the same code is generated
the 33.2 times speed-up for the lists of size 2048. The results show each time.
that with increasing list length, the speed-up converges asymptoti- The GPU is capable of running more than 1000 threads simulta-
cally. One possible reason for this asymptotic convergence is due neously, so a second benchmark is performed that involves sorting
to the fact that sorting 1000 lists in parallel does not fully exploit 5000 lists rather than 1000 lists. The purpose of this benchmark is
the parallel nature of the GPU. to examine diferent usage scenarios that may allow GPUMap to
Figure 7 shows a breakdown of the run times of diferent stages take better advantage of the parallel processing power of the GPU.
of GPUMap for the diferent input data sizes for the bubble sort The GPU is able to achieve a much better speedup when sorting
benchmark. Similarly to the n-body benchmark, due to the fact that these 5000 lists in parallel, as shown in Figure 8.
8
 speed-up, 13 times, occurs when there are 10000 lists of size 10000
being sorted.

Figure 8: Speed-up using GPUMap to sort 5000 lists using

bubble sort

Figure 9: Speed-up using GPURDD to sort lists of size 10000

using shellsort
With a larger number of lists to operate on in parallel, the GPU
is able to use more of its available threads simultaneously. As a
The reason this benchmark results in smaller speed-ups than
result, the maximum possible speed-up from increasing data size
the bubble sort benchmark is because this algorithm has better
increases from about 34 times in the benchmark with 1000 lists, to
time complexity than bubble sort. In addition, the number of lists
about 145 times in the benchmark with 5000 lists.
is varied rather than the number of elements in each list, causing
Due to the fact that bubble sort is an O(n 2 ) algorithm, using
larger numbers of lists to linearly increase the amount of work to
bubble sort with GPUMap to sort multiple lists simultaneously pro-
do. This means that the kernel duration is not able to outweigh
duces considerable performance improvements, as the processing
the frst call and serialization stages’ durations. For bubble sort’s
time is not overshadowed by the serialization time. However, the
benchmarks involving larger data sets, the kernel had a longer
performance improvement seems to be somewhat dependent on
duration than serialization, thus resulting in a larger impact on the
the data size, and unfortunately, reshaping the data into a shape
overall duration. However, on the shell sort tests, the total duration
that maximizes performance may not be possible.
was impacted by the frst call, serialization, and deserialization
stages more signifcantly than by the kernel run stage, as depicted
5.3 Shell Sort Benchmark in Figure 10.
The shell sort test benchmarks the sorting of multiple lists in Overall, GPURDD may provide a slight performance improve-
parallel, similarly to the bubble sort. However, this test uses the ment over a normal RDD when using an algorithm such as shell sort
shell sort algorithm, which has a worst case time complexity of with a large enough data set. However, performance improvement
O(nloд2n) [11]. In addition, this test designed for use with GPURDD is dependent on the shape of the data, which can make depending
rather than GPUMap, and the list count is varied rather than the on GPURDD for performance improvements less feasible. In addi-
list size. Varying the list count rather than the list size makes this tion, due to the fact that in this experiment the work to be done
parallel list sort an O(m) where m is the number of lists. is O(m) on the number of lists, the serialization costs cannot be
Prior to running the benchmark, a warm up is performed using outweighed with processing time.
100 lists of length 100 partitioned into 10 partitions. Then, the
tests are run in order measure the performance of sorting 100, 1000, 6 CONCLUSIONS AND FUTURE WORK
10000, and 100000 lists of length 10000, partitioned into 10 partitions. This paper presents GPUMap, a Python map function that aims to
The benchmark measures the time taken for the each set of lists to be allow programmers to GPU-accelerate certain types of programs
sorted using both RDD’s foreach method and GPURDD’s foreach with no extra efort or knowledge of GPU computing.. GPUMap
method. As shown in Figure 9, for this benchmark, GPURDD’s works with normal, object-oriented Python code and does not re-
foreach performs at least as well as RDD’s foreach on each of quire users to do any serialization or write kernels. This paper also
these tests, with the exception of the smallest data set due to the presents GPURDD, which is a type of Spark RDD that incorporates
added overhead of GPUMap. The sorting smallest data set, 100 lists GPUMap into its map, filter, and foreach methods to provide
of size 10000, results in poor performance because GPUMap is not simplifed GPU acceleration in Apache Spark.
able to take full advantage of the GPU as there are only 100 lists GPUMap performs automatic serialization, automatic code gener-
to sort, so only 100 GPU threads can be used. The most signifcant ation, automatic kernel execution, and automatic deserialization in
9
 GPURDD incorporates GPUMap, all of the limitations of GPUMap
carry over to GPURDD.

REFERENCES
[1] Jefrey Dean and Sanjay Ghemawat. 2008. MapReduce: simplifed data processing
on large clusters. Commun. ACM 51, 1 (2008), 107–113.
[2] Gary Frost and Mohammed Ibrahim. 2015. Aparapi Documentation. Technical
Report. AMD Open Source Zone. https://aparapi.github.io/http://developer.amd.
com/tools-and-sdks/open-source/
[3] Andreas Klöckner, Nicolas Pinto, Yunsup Lee, B. Catanzaro, Paul Ivanov, and
Ahmed Fasih. 2012. PyCUDA and PyOpenCL: A Scripting-Based Approach
to GPU Run-Time Code Generation. Parallel Comput. 38, 3 (2012), 157–174.
https://doi.org/10.1016/j.parco.2011.09.001
[4] Siu Kwan Lam, Antoine Pitrou, and Stanley Seibert. 2015. Numba: A LLVM-based
Python JIT Compiler. In Proceedings of the Second Workshop on the LLVM Compiler
Infrastructure in HPC (LLVM ’15). ACM, New York, NY, USA, Article 7, 6 pages.
https://doi.org/10.1145/2833157.2833162
[5] Y. Lin, S. Okur, and C. Radoi. 2012. Hadoop+Aparapi: Making heterogenous MapRe-
duce programming easier. Technical Report. hgpu.org. http://www.semihokur.
com/docs/okur2012-hadoop_aparapi.pdf
[6] NVIDIA Corporation. 2017. CUDA C Programming Guide. Technical Report.
NVIDIA Corporation. http://docs.nvidia.com/cuda/cuda-c-programming-guide
[7] John D Owens, Mike Houston, David Luebke, Simon Green, John E Stone, and
Figure 10: Stage duration using GPURDD to sort lists of size James C Phillips. 2008. GPU computing. Proc. IEEE 96, 5 (2008), 879–899.
10000 using shell sort [8] Ivan Pachev. 2017. GPUMap: A Transparently GPU-Accelerated Map Function.
Master’s thesis. California Polytechnic State University.
[9] Philip C Pratt-Szeliga, James W Fawcett, and Roy D Welch. 2012. Rootbeer:
Seamlessly using GPUs from Java. In High Performance Computing and Communi-
order to attempt to provide the programmer with a GPU-accelerated cation & 2012 IEEE 9th International Conference on Embedded Software and Systems
(HPCC-ICESS), 2012 IEEE 14th International Conference on. IEEE, Liverpool, United
map function that does not require any extra efort to use. Kingdom, 375–380.
GPUMap achieves a considerable performance improvement for [10] Python Software Foundation. 2017. Data Model. Technical Report. Python
certain types of programs. For compatible algorithms that have Software Foundation. https://docs.python.org/3/reference/datamodel.html
[11] Robert Sedgewick. 1996. Analysis of Shellsort and Related Algorithms. In Proceed-
considerably larger time complexity than O(n) and a large enough ings of the Fourth Annual European Symposium on Algorithms (ESA ’96). Springer-
data set, GPUMap may provide performance improvements. Dur- Verlag, London, UK, UK, 1–11. http://dl.acm.org/citation.cfm?id=647906.739656
[12] Oren Segal, Philip Colangelo, Nasibeh Nasiri, Zhuo Qian, and Martin Margala.
ing benchmarking of GPUMap using the O(n 2 ) n-body algorithm, 2015. SparkCL: A Unifed Programming Framework for Accelerators on Hetero-
GPUMap was able to produce a speed up of 249 times on the largest geneous Clusters. CoRR abs/1505.01120 (2015). http://arxiv.org/abs/1505.01120
data set. However, for algorithms with O(n) time complexity or [13] Artjoms Šinkarovs, Sven-Bodo Scholz, Robert Bernecky, Roeland Douma, and
Clemens Grelck. 2014. SaC/C formulations of the all-pairs N-body problem and
better, GPUMap will likely not yield any considerable speed-ups their performance on SMPs and GPGPUs. Concurrency and Computation: Practice
due to O(n) serialization complexity. and Experience 26, 4 (2014), 952–971.
GPURDD was created to incorporate the simplifed GPU-acceleration [14] J. A. Stuart and J. D. Owens. 2011. Multi-GPU MapReduce on GPU Clusters. In
2011 IEEE International Parallel Distributed Processing Symposium. 1068–1079.
provided by GPUMap into Apache Spark. GPURDD’s map and https://doi.org/10.1109/IPDPS.2011.102
foreach methods use GPUMap to apply a given function to each [15] Yonghong Yan, Max Grossman, and Vivek Sarkar. 2009. JCUDA: A programmer-
friendly interface for accelerating Java programs with CUDA. In Euro-Par 2009
item in a partition. In the case of the filter method, GPUMap is Parallel Processing: 15th International Euro-Par Conference, Delft, The Netherlands,
used to apply the fltering function to each item in a partition to August 25-28, 2009. Proceedings. Springer, Springer Berlin Heidelberg, Berlin,
determine whether each element should be kept. The elements that Heidelberg, 887–899. https://doi.org/10.1007/978-3-642-03869-3_82
[16] Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma,
should be not kept are then pruned outside of GPUMap. Murphy McCauly, Michael J. Franklin, Scott Shenker, and Ion Stoica. 2012. Re-
silient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster
Computing. In Presented as part of the 9th USENIX Symposium on Networked Sys-
6.1 Future Work tems Design and Implementation (NSDI 12). USENIX, San Jose, CA, 15–28. https:
There are some Python language features, data structures, and built- //www.usenix.org/conference/nsdi12/technical-sessions/presentation/zaharia
[17] Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, and Ion
in functions that are unsupported in code translated by GPUMap, Stoica. 2010. Spark: Cluster Computing with Working Sets. HotCloud 10 (2010),
primarily because GPUMap does not make use of CUDA’s thread- 10–10.
level dynamic allocation as it does not perform well when many
threads attempt to allocate memory simultaneously. This means
that variable length data structures such as lists, dicts, and strings
are unsupported by GPUMap. However, GPUMap supports limited
usage of lists that are included as input list elements or closure
variables. By using an alternative thread-level dynamic allocation
scheme, it may be possible to incorporate dynamic allocation into
GPUMap so that many more Python features can be implemented.
There are further performance improvements that can be made
to GPUMap by caching input lists and closure variables and par-
allelizing the serialization process, which may help GPUMap per-
form better overall by decreasing the serialization time. Because
10