PDF Parallel Programming For Modern High Performance Computing Systems Czarnul Download
PDF Parallel Programming For Modern High Performance Computing Systems Czarnul Download
PDF Parallel Programming For Modern High Performance Computing Systems Czarnul Download
com
https://textbookfull.com/product/parallel-
programming-for-modern-high-performance-computing-
systems-czarnul/
https://textbookfull.com/product/parallel-and-high-performance-
computing-meap-v09-robert-robey/
textbookfull.com
https://textbookfull.com/product/high-performance-computing-for-
geospatial-applications-wenwu-tang/
textbookfull.com
https://textbookfull.com/product/monitoring-multimode-continuous-
processes-a-data-driven-approach-marcos-quinones-grueiro/
textbookfull.com
Detective Michael Angel Box Set Books 1 7 Roger Silverwood
https://textbookfull.com/product/detective-michael-angel-box-set-
books-1-7-roger-silverwood/
textbookfull.com
https://textbookfull.com/product/options-futures-and-other-
derivatives-eleventh-edition-rental-john-c-hull/
textbookfull.com
https://textbookfull.com/product/development-of-implantable-
electronics-as-novel-approaches-to-obstructive-sleep-apnea-jungmin-
seo/
textbookfull.com
https://textbookfull.com/product/songs-of-ukraina-with-ruthenian-
poems-florence-randal-livesay/
textbookfull.com
New Models of Financing and Financial Reporting for
European SMEs: A Practitioner's View Eva Kaili
https://textbookfull.com/product/new-models-of-financing-and-
financial-reporting-for-european-smes-a-practitioners-view-eva-kaili/
textbookfull.com
Parallel Programming
for Modern
High Performance
Computing Systems
Parallel Programming
for Modern
High Performance
Computing Systems
Paweł Czarnul
Gdańsk University of Technology, Poland
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access
www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc.
(CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization
that provides licenses and registration for a variety of users. For organizations that have been granted
a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
To my daughter Ala
Contents
1.1 INTRODUCTION 1
1.2 FROM PROBLEM TO PARALLEL SOLUTION –
DEVELOPMENT STEPS 2
1.3 APPROACHES TO PARALLELIZATION 4
1.4 SELECTED USE CASES WITH POPULAR APIS 6
1.5 OUTLINE OF THE BOOK 7
vii
viii Contents
Index 297
List of figures
xiii
xiv List of figures
2.1 Performance of the first cluster on the TOP500 list over time 20
2.2 The number of cores of the first cluster on the TOP500 list
over time 20
2.3 CPU clock frequency of the first cluster on the TOP500 list
over time 21
2.4 Performance to power consumption ratio of the first cluster on
the TOP500 list over time 21
xvii
xviii List of tables
xix
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
xx List of listings
Parallel computing systems have recently become more and more accessible to
a wide range of users. Not only programmers in high performance computing
centers but also a typical consumer can now benefit from high performance
computing devices installed even in desktop computers. The vast majority of
new computers sold today feature multicore CPUs and GPUs which can be
used for running parallel programs. Such usage of GPUs is often referred to
as GPGPU (General Purpose Computations on Graphics Processing Units).
Among devices announced by manufacturers are, for instance, a 7th generation
Intelr Core™ i7-7920HQ CPU that features 4 cores with HyperThreading for
8 logical processors clocked at 3.1GHz (up to 4.1GHz in turbo mode) and
a TDP (Thermal Design Power) of 45W. AMD Ryzen™ 7 1800X features 8
cores for 16 logical processors clocked at 3.6 GHz (4 GHz in turbo mode)
and a TDP of 95W. A high end desktop Intel Core i9-7900X CPU features
10 cores with HyperThreading for 20 logical processors clocked at 3.3GHz
(up to 4.3GHz in turbo mode) and a TDP of 140W. NVIDIAr Titan X,
based on the NVIDIAr Pascal™ architecture, features 3584 CUDAr cores
and 12GB of memory at the base clock of 1417MHz (1531MHz in boost)
with a power requirement of 250W. Workstations or servers can use CPUs
such as Intelr Xeonr Scalable processors such as Intel Xeon Platinum 8180
processor that features 28 cores and 56 logical processors clocked at 2.5 GHz
(up to 3.8GHz in turbo mode) and a power requirement of 205W or Intel
Xeon E5-4669v4 with 22 cores and 44 logical processors clocked at 2.2 GHz (3
GHz in turbo mode) and a TDP of 135W. AMD Opteron™ 6386 SE features
16 cores clocked at 2.8 GHz (3.5 GHz in turbo mode) with a TDP of 140 W.
High performance oriented GPUs include NVIDIAr Teslar P100, based on
the Pascal architecture, with 3584 CUDA cores at the base clock of 1480MHz
in boost and with a power requirement of 250W as well as NVIDIA Tesla
V100 with 16 GB HBM2 memory, 5120 CUDA cores clocked at 1455MHz
in boost and with a power requirement of 300W. AMD FirePro™ W9100
features 2816 Stream Processors, 32GB or 16GB GDDR5 GPU memory with
a power requirement of 275W. High performance oriented machines can use
coprocessors such as Intelr Xeon Phi™ x100 7120A with 61 cores clocked at
1.238GHz and a TDP of 300W or e.g. Intel Xeon Phi x200 7250 processors
with 68 cores clocked at 1.4GHz with a TDP of 215W. As it was the case
in the past and is still the case today, computer nodes can be interconnected
together within high performance computing clusters for even greater compute
xxiii
xxiv Preface
The target audience of this book are students, programmers, domain spe-
cialists who would like to become acquainted with:
[58]Genealogia Deorum.
[59]Baldelli.
[60]Filippo Villani.
[61]Geneal. Deor.
[62]Geneal. Deor.
[63]Ibid.
[64]Tiraboschi.
[65]Filocopo.
[66]This lady Mary cannot be the princess Mary, an acknowledged natural
daughter of king Robert. The latter was beheaded during the troubles at Naples, a
year after Boccaccio's death. The poems of Boccaccio declare that he outlived his
lady Mary, Fiammetta, as he called her, many years; and his writings give proof
that her royal and illegitimate origin was always preserved a secret.
[67]La Fiammetta.
[68]Rime.
[69]Ameto.
[70]Baldelli.
[71]Petrarch's Letters.
[72]This singular circumstance is not noticed by Petrarch in any of his letters. Did
the Florentines act thus to punish him for his journey to Avignon, at the time they
had invited him to take up his abode among them? Yet, on another occasion, the
citizens petitioned the pope to give the poet a benefice within their walls, and so
induce him to inhabit their city. Perhaps the expression used in Boccaccio's letter is
ironical.
[73]Guignenè.
[74]It is not creditable to the learning of those times to learn, that the libraries of
these two great revivers of knowledge were lost to the world soon after their
deaths. Boccaccio's, it is true, was destroyed by an accident, being burnt when the
convent to which he had left it was consumed by fire. But Petrarch's mouldered
away in the palace given by the republic of Venice for its reception and
preservation, so that dusty fragments were afterwards found to be all that
remained of the venerable parchments which the laureate had expended so much
time and labour in collecting.
[75]Baldelli.
[76]It is unknown who was the mother of this child, or grandchild, who died so
young. Boccaccio had, besides, one son established at Florence, whom he does
not mention in his will, but who presided at his funeral, and erected a tomb over
his remains.
[77]Baldelli, Cod. San. Epist. I.
(considered as a poet);
"Welcome, May,
And the rustic banner," &c.—
is the prettiest and most spirited song for May ever written. His
processions and masquerades afforded also subjects for verse.
Bands of people paraded the city in character, personating triumphs,
or exhibitions of the arts; and Lorenzo wrote songs, which they
chanted as they passed along. It is singular, that, free and energetic
as the Florentines were, yet the songs composed for them never
spoke of liberty, but turned upon love only: love was all their theme
—love that was often licentiousness, and yet described with such
truth and beauty, as must have tended greatly to enervate, and even
to vitiate, the various persons that formed these gay companies.
Lorenzo's canzoni are tainted with this defect.
Lorenzo was a faithful and kind, though not a fond husband. His
feelings were always held in discipline by him; and if he were too
sensitive to the influence of beauty, yet his actions were all regulated
by that excellent sense of justice and duty which is his admirable
characteristic. There are some elegiac stanzas preserved of his,
which prove that he suffered at one time the struggles and errors of
passion, and was subdued by it to other thoughts than those which
his reason approved. How different is this poem to those addressed
to Lucretia Donati. There is no Platonic refinement, no subtlety, no
conceit, no imitation of Petrarch; its diction is clear and sweet; truth
and strength of feeling animate each expression; it bears the stamp
of heartfelt sincerity, and is adorned by all the delicacy which real
passion inspires. "Ah!" he exclaims, "had we been joined in
marriage! Had you been earlier born, or had I come later into the
world!" These stanzas are even left unfinished, and probably were
concealed, as revealing a secret which it would have been fatal to
have discovered to the world.
Besides the animated and gay songs, and choruses, in which
Lorenzo is unrivalled, he wrote several descriptive poems: one long
one relates the history of how his favourite country house, named
Ambra, was carried away by the overflowing of the Ombrone. He
figures the villa to be a nymph, of whom the river god is enamoured,
and, like one of Ovid's heroines, she falls a victim to his pursuit. The
descriptions in this poem are lively, true, and graceful. The "Caccia di
Falcone" gives a spirited detail of the disasters that befall falconers:
he bring in several of his friends by name. "Where is Luigi Pulci," he
cries, "that we do not hear him? He is gone before in that grove, for
some whim has seized him, and he has retreated to meditate a
sonnet."
Lorenzo died at the early age of forty-four, of a painful and April
8.
inexplicable disorder, which, attacking his stomach, gave rise
to the idea that he was poisoned. He was considerate and 149
2
affectionate to the last; endeavouring to impress his system of
policy on his son's mind, and exerting himself to lighten the grief of
those around him. Potents and wonders followed his death, which
even Machiavelli, then a very young man, deemed miraculous. He
was universally lamented; and the downfall of his family, which
occurred soon after, through the misconduct of his eldest son, Piero,
renewed the grief of the friends who survived him.
MARSIGLIO FICINO
ANGELO POLIZIANO
While singing these verses, after Lorenzo's death, afflicted at the sad
loss they commemorated, and by the adverse events which followed,
a spasm of grief seized him, his heart suddenly broke from excess of
feeling, and he died on the spot. He died on the 24th of September,
1494, having just completed his 40th year, and having survived his
illustrious friend little more than two years.
BERNARDO PULCI
LUCA PULCI