Introduction To HPC: Content and Definitions

Introduction to HPC
content and definitions

Jan Thorbecke, Section of Applied Geophysics
Delft
University of
Technology
Challenge the future

Motivation and Goal
Get familiar with hardware building blocks, how they operate, and
how to make use of them in writing efficient programs.
Understand the function of an Operating System (Linux).
Participants should be able to write, compile and run numerical

code efficiently on standard hardware.
Start to think how to solve problems in Parallel (try to avoid

thinking in sequential loops).
During the course keep in mind a numerical problem which you

would like to implement/improve.
2
Learning Outcomes
Understand the architecture of modern CPUs and how this
architecture influences the way programs should be written.
Optimize all aspects in the processes of programming,

compilation, starting program, running program by OS, executing
(parallel) instructions by CPU to writing output to disk.
Write numerical software, which exploits the memory hierarchy of

a CPU to obtain a code with close to optimal performance.
Analyze an existing program for OpenMP and MPI parallelization

possibilities.
Evaluate the possibilities of accelerators to speed up

computational work.
3
Organization
Dates: 7, 12, 21 May and 11, 18 June 2015
Time: 13:30 17:00
Points: 2
Examination: attendance, exercises and discussions during the

course. For MSc students multiple-choice exam with discussion
afterwards.
4
Schedule and rooms
CITG Building:
Stevinweg 1
2628 CN Delft
7 May room 2.02

12 May room 2.99
21 May room 2.02
11 June room 2.02
18 June room 2.02
5
What is HPC
High-performance computing (HPC) uses supercomputers and computer

clusters to solve advanced computation problems. Today, computer systems in
the teraflops-region are counted as HPC-computers. (Wikipedia)
A list of the most powerful high performance computers can be found on the
TOP500 list. The TOP500 list ranks the world's 500 fastest high performance
computers as measured by the HPL benchmark.
For this course: Scientific programs written for optimal use of

computational hardware.
6
Four Components Computer System
user user user user
1 2 3 n
IO fdelmodc compiler text
system and application programs
operating System
hardware
7
Computer System Structure
Computer system can be divided into four components
Hardware provides basic computing resources

CPU, memory, I/O devices
Operating system
Controls and coordinates use of hardware among various applications
and users
Application programs to solve the computing problems of users

Word processors, compilers, web browsers, database systems, games
Users
People, machines, other computers
8
Contents
Introduction (1)
Hardware (1,2)
OS (2)
Programming (2/3)
Optimization (3)
Multi-processor hardware(3/4)
Writing Parallel programs (4/5)
Parallel programming + wrap-up and discussion (5)
9
Part 1
Components of a computer system
addressing
memory hardware
Memory Hierarchy
types of cache mappings
cache levels
virtual memory, TLB
Processors
out of order
pipelining
branches
multi-core cache-coherency
SSE2/3
Modern processors
SansyBridge, Power7, Interlagos, GPU
Future trends in hardware
10
Part 2
Operating System
IO management
Processes
IPC, RPC
Virtual memory management

allocation
dynamic static
page replacement policies
File systems
storage, RAID
Linux, Unix, OSX, XP
11
Part 3
User environment
login, shells
Compiling and linking

compile stages
Makefiles
Number representations
IEEE 754 standard
Programming languages
C, Fortran
profiling and debugging
numerical libraries
12
Part 4
Computer Performance
Amdahls law
standard benchmarks Spec, Linpack
Optimization
cache blocking
loops
examples
13
Part 5
Programming for parallel systems
Multi-processor hardware
Classification
HPC hardware
Programming for parallel systems

OpenMP
MPI
examples
Finish the last parts of the course
Followed by questions, discussion and evaluation of exercises
14
The Exercises
Exercises are set up to get familiar with concepts and ideas. If

you already have enough experience with some parts in the
exercise you can skip those parts.
Exercises are simple and straightforward, no programming skills

are needed, and consists of compilation and running of small
programs. Every exercise comes with a step by step procedure to
follow.
The exercises are hopefully fun to do and are not obligatory to

pass the course, but highly recommended.
15
The slides
There are quite a few of them...
Not all of them will be discussed
Sometimes techniques and explained in two different ways, one

will be used in the lectures, the other one can be used to
understand the concepts better.
At the moment there are no lecture notes and the detailed slides
try to compensate that.
16
Definitions and Terms
bit = binary number with states 0 or 1

byte = 8 bits
word = 4 or 8 bytes (the number of bits used in a register)
Kbit = 1000 bits, used for communication speeds
KiB = 1024 (210) Bytes, used for memory and storage
MiB = 1048576 (220) Bytes
flops = floating point operations per second
CPU = Central Processing Unit
FPU = Floating Point Unit
ALU = Arithmetic Logic Unit
AGU = Address Generation Unit
http://physics.nist.gov/cuu/Units/binary.html
17
Definitions
Latency: Latency is a time delay between the moment

something is initiated, and the moment one of its effects
begins or becomes detectable. The word derives from the
fact that during the period of latency the effects of an action
are latent, meaning "potential" or "not yet observed".
Bandwidth: a data transmission rate; the maximum amount

of information (bits/second) that can be transmitted along a
channel.
flops: number of floating point operations (addition

multiplication) per second.
18
Clock
Most modern CPUs are driven by a clock.
The CPU consists internally of logic and memory (flip flops).
When the clock signal arrives, the flip flops take their new
value and the logic then requires a period of time to decode
the new values.
Then the next clock pulse arrives and the flip flops again take
their new values, and so on.
19
Feedback
Positive and negative feedback about the course is highly

appreciated.
You can always send me an e-mail or come to my desk in

room 3.12 (CITG Building on Tuesdays and Thursdays only).
20
Book References
Computer Architecture , A Quantitative Approach
Fifth Edition, John L. Hennessy, David A. Patterson 2011 Morgan Kaufmann
The C Programming Language

(2nd edition), Kernighan & Ritchie, 1988 Prentice Hall
Modern Fortran Explained

(Numerical Mathematics and Scientific Computation)
Michael Metcalf, John Reid , Malcolm Cohen, 2011 Oxford University Press
Fortran 90 Programming
T.M.R. Ellis, Ivor R. Phillips, Thomas M. Lahey, 1994 Addison Wesley
Parallel Programming With MPI

Peter Pacheco, 1996 Morgan Kaufmann
21
References on internet
http://arstechnica.com/index.ars
http://www.tomshardware.co.uk/
http://insidehpc.com/
http://www.gpgpu.org/
http://www.theregister.co.uk/
wikipedia.org
22

Introduction To HPC: Content and Definitions

Uploaded by

Copyright:

Available Formats

Introduction To HPC: Content and Definitions

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To HPC: Content and Definitions

Uploaded by

Copyright:

Available Formats

Introduction to HPC

content and definitions

Challenge the future

Understand the function of an Operating System (Linux).

Participants should be able to write, compile and run numerical

Start to think how to solve problems in Parallel (try to avoid

During the course keep in mind a numerical problem which you

Optimize all aspects in the processes of programming,

Write numerical software, which exploits the memory hierarchy of

Analyze an existing program for OpenMP and MPI parallelization

Evaluate the possibilities of accelerators to speed up

Dates: 7, 12, 21 May and 11, 18 June 2015

Time: 13:30 17:00

Examination: attendance, exercises and discussions during the

7 May room 2.02

High-performance computing (HPC) uses supercomputers and computer

For this course: Scientific programs written for optimal use of

IO fdelmodc compiler text

system and application programs

Hardware provides basic computing resources

Application programs to solve the computing problems of users

Virtual memory management

Linux, Unix, OSX, XP

Compiling and linking

Programming for parallel systems

Finish the last parts of the course

Followed by questions, discussion and evaluation of exercises

Exercises are set up to get familiar with concepts and ideas. If

Exercises are simple and straightforward, no programming skills

The exercises are hopefully fun to do and are not obligatory to

There are quite a few of them...

Not all of them will be discussed

Sometimes techniques and explained in two different ways, one

bit = binary number with states 0 or 1

Latency: Latency is a time delay between the moment

Bandwidth: a data transmission rate; the maximum amount

flops: number of floating point operations (addition

Most modern CPUs are driven by a clock.

The CPU consists internally of logic and memory (flip flops).

Positive and negative feedback about the course is highly

You can always send me an e-mail or come to my desk in

The C Programming Language

Modern Fortran Explained

Parallel Programming With MPI

You might also like