Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
21 views

Lecture-4 Parallel Programming Model

Uploaded by

Saad Tayef
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
21 views

Lecture-4 Parallel Programming Model

Uploaded by

Saad Tayef
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 14

Course No: CSE 4117

Course Name: Parallel and


Distributed Processing
Parallel Programming Model

 Parallel programming model exists an abstraction above hardware and memory


architectures. It’s a set of program abstractions for fitting parallel activities from the
application to the underlying parallel hardware. It spans over different layers: applications,
programming languages, compilers, libraries, network communication, and I/O systems.

Shared Memory Model: Multiple processes share common memory space

Distributed Memory Model :The user makes call to libraries to explicitly share information
between processors.
Threads Model: A single process having multiple (concurrent) execution paths

Data Parallel Model: Data partitioning determines parallelism


Parallel Programming Model
Shared Memory Model (Without Thread)
• In this programming model, processes/tasks share a common address space, which they read
and write to asynchronously.
• Various mechanisms such as locks / semaphores are used to control access to the shared
memory, resolve contentions and to prevent race conditions and deadlocks.
• This is perhaps the simplest parallel programming model.
Parallel Programming Model
Shared Memory Model (Without
Thread)
• The processor may not have a private program or
data memory. A common program and data are
stored in the main memory. This is accessed by all
processors.
• Each processor is assigned a different part of
the program and the data. The main program
creates separate process for each processor.
• The process is allocated to the processor along
with the required data. These process are executed
indecently on different processors.
• After the execution, all the processors have to
rejoin the main program
Parallel Programming Model
Advantages of shared memory model
• Data "ownership" is lacking, so there is no need to specify explicitly the communication of
data between tasks.
• All processes have equal access to shared memory.

• Program development becomes simple.

Disadvantages of shared memory model


• Different techniques need to use to block the access of the same data among different
processors.
• Users will not understand where his variables use stored.
Parallel Programming Model
Distributed Memory Model
• A set of tasks that use their own local memory during computation.
• Multiple tasks can reside on the same physical machine and/or across an arbitrary number of machines.
• Tasks exchange data through communications by sending and receiving messages.
• Data transfer usually requires cooperative operations to be performed by each process. For example, a
send operation must have a matching receive operation.
Parallel Programming Model
Distributed Memory Model
Parallel Programming Model

Advantages of Distributed memory model


• Data can be stored anywhere.

• Communication between processes are simple.

• Many Manage Passing Interfaces (MPI) are available.

Disadvantages of Distributed memory model


• Programmers should take care that there is a receive function for every send function.

• If any dependent process is quit or stop then the other processes will stop working.
• What are the advantages and disadvantages of using a shared memory
model compared to a distributed memory model?

• How can race conditions and deadlocks be avoided in a shared


memory model?
Parallel Programming Model
Thread Model
• A thread is defined as a short sequence of
instructions, within a process.
• This programming model is a type of shared memory
programming.
• In the threads model of parallel programming, a single
"heavy weight" process can have multiple "light
weight", concurrent execution paths called threads.
• Each thread has local data, but also, shares the
common data.
• Threads communicate with each other through global
memory (updating address locations).
• This requires synchronization constructs to ensure
that more than one thread is not updating the same
global address at any time.
Parallel Programming Model
Thread Model
• Different threads can be
executed on same processor or
on different processors.

• If the threads are executed on


same processor, then the
processor switches between the
threads in a random fashion.

• If the threads are executed on


different processors, they are
executed simultaneously.
Parallel Programming Model

Advantages of Thread Model


• Programmer need not have to worry about parallel processing.

Disadvantages of Thread Model


• Care must be taken that no two threads update the shared resources
simultaneously.
Parallel Programming Model
Data Parallel Model
• Data parallelism is one of the simplest forms of parallelism. Here data set is organized into a common structure.
It could be an array. It may also be referred to as the Partitioned Global Address Space (PGAS) model.

• The data parallel model demonstrates the following characteristics:

• Address space is treated globally

• Most of the parallel work focuses on performing operations on a data set. The data set is typically organized
into a common structure, such as an array.

• A set of tasks work collectively on the same data structure, however, each task works on a different partition
of the same data structure.

• On shared memory architectures, all tasks may have access to the data structure through global memory.

• On distributed memory architectures, the global data structure can be split up logically and/or physically across
tasks.
Parallel Programming Model
 Example of Data Parallel Model
Int x [100],
For(i=0;i<100; i++)
y [100], z[i] = x[i] + y[i]
z [100]

Processor 1 Processor 2 Processor 3 Processor 4

For(i=0;i<25; i++) For(i=25;i<50; i++) For(i=50;i<75; i++) For(i=75;i<100; i++)


z[i] = x[i] + y[i] z[i] = x[i] + y[i] z[i] = x[i] + y[i] z[i] = x[i] + y[i]

You might also like