CS4230 Parallel Programming: Mary Hall August 21, 2012
CS4230 Parallel Programming: Mary Hall August 21, 2012
CS4230 Parallel Programming: Mary Hall August 21, 2012
Lecture 1: Introduction
Mary Hall
August 21, 2012
08/21/2012 CS4230 1
Course Details
• Time and Location: TuTh, 9:10-10:30 AM, WEB L112
• Course Website
- http://www.eng.utah.edu/~cs4230/
• Instructor: Mary Hall, mhall@cs.utah.edu,
http://www.cs.utah.edu/~mhall/
- Office Hours: Mon 11:00-11:30 AM; Th 10:45-11:15 AM
• TA: TBD
- Office Hours: TBD
• Textbook
- “An Introduction to Parallel Programming,”
Peter Pacheco, Morgan-Kaufmann Publishers, 2011.
- Also, readings and notes provided for other topics as needed
08/21/2012 CS4230 2
Administrative
• Prerequisites:
- C programming
- Knowledge of computer architecture
- CS4400 (concurrent ok for seniors)
• Please do not bring laptops to class!
• Do not copy solutions to assignments from the internet
(e.g., wikipedia)
• Read Chapter 1 of textbook by next lecture
• First homework handed out next time
08/21/2012 3
CS4230
Basis for Grades
• 35% Programming projects (P1,P2,P3,P4)
• 20% Written homeworks
• 5% Participation (in-class assignments)
• 25% Quiz and Final
• 15% Final project
08/21/2012 CS4230 4
Today’s Lecture
• Overview of course
• Important problems require powerful computers …
- … and powerful computers must be parallel.
- Increasing importance of educating parallel
programmers (you!)
- Some parallel programmers need to be performance
experts – my approach
• What sorts of architectures in this class
- Multimedia extensions, multi-cores, SMPs, GPUs,
networked clusters
• Developing high-performance parallel applications
• An optimization perspective
08/21/2012 5
CS4230
Course Objectives
• Learn how to program parallel processors and
systems
- Learn how to think in parallel and write correct
parallel programs
- Achieve performance and scalability through
understanding of architecture and software mapping
Clock speed
flattening
sharply
Moore’s Law:
Gordon Moore (co-founder of Intel) predicted in 1965 that the transistor
density of semiconductor chips would double roughly every 18 months.
08/21/2012 CS4230 8
The Multi-Core or Many-Core Paradigm Shift
08/21/2012 CS4230 11
Example: Global Climate Modeling Problem
• Problem is to compute:
f(latitude, longitude, elevation, time)
temperature, pressure, humidity, wind velocity
• Approach:
- Discretize the domain, e.g., a measurement point every 10 km
- Devise an algorithm to predict weather at time t+dt given t
• Uses:
- Predict major events,
e.g., El Nino
- Use in setting air
emissions standards
Source: http://www.epm.ornl.gov/chammp/chammp.html
08/21/2012 CS4230 12
Some Characteristics of Scientific Simulation
08/21/2012 CS4230 13
Example of Discretizing a Domain
One
Another
processor
processor
computes
computes
this part
this part in
parallel
08/21/2012 CS4230 14
Parallel Programming Complexity: An Analogy
• Enough Parallelism (Amdahl’s Law)
• Parallelism Granularity
• Independent work between coordination points
• Locality
- Perform work on nearby data
• Load Balance
- Processors have similar amount of work
• Coordination and Synchronization
- Who is in charge? How often to check in?
08/21/2012 CS4230 15
Course Goal
• Most people in the research community agree that
there are at least two kinds of parallel programmers
that will be important to the future of computing
• Programmers that understand how to write software,
but are naïve about parallelization and mapping to
architecture (Joe programmers)
• Programmers that are knowledgeable about
parallelization, and mapping to architecture, so can
achieve high performance (Stephanie programmers)
• Intel/Microsoft say there are three kinds (Mort, Elvis
and Einstein)
• This course is about teaching you how to become
Stephanie/Einstein programmers
Course Goal
• Why OpenMP, Pthreads, MPI and CUDA?
• These are the languages that Einstein/Stephanie
programmers use.
• They can achieve high performance.
• They are widely available and widely used.
• It is no coincidence that both textbooks I’ve used
for this course teach all of these except CUDA.