TM275: Parallel and

Distributed Systems
Dr. Eng. Khouloud Samrouth & Dr. Rawad Abdulghafor

TM275 - Parallel Programming with Python

1
Meeting 2#
Parallel Programming
with Python
Prerequisite

TM275 - Parallel Programming with Python

 Installation

Make sure to
check this option

TM275 - Parallel Programming with Python

4
Installation
• Open the CMD
• Verify if python is added to “variables env. Path”

TM275 - Parallel Programming with Python

5
Installation
• Add python to “variables env. Path”

TM275 - Parallel Programming with Python

6
Python IDLE

TM275 - Parallel Programming with Python

7
Python IDLE Editor
• IDLE development environment:
• Shell for interactive evaluation.
• Text editor with color-coding and smart indenting for creating python files.
• Menu commands for changing system settings and running files.
• Search for “IDLE” from the start menu

TM275 - Parallel Programming with Python

8
.py

TM275 - Parallel Programming with Python

9
Run file.py
• py path_to_file\filename.py

TM275 - Parallel Programming with Python

10
Introduction

TM275 - Parallel Programming with Python

Parallel Programming
• Parallel Programming consists of implementing digital architectures allowing
information to be processed simultaneously.
• These techniques aim to carry out the greatest number of operations in the
shortest possible time.
• A parallel program is divided into several sequential tasks running
simultaneously, usually called processes.

TM275 - Parallel Programming with Python

12
Process
• A process is a program being executed by a computer

TM275 - Parallel Programming with Python

13
Parent Process – Child Process

Execute independtly Have their own memory space

TM275 - Parallel Programming with Python

14
Parallel Programming
with Python

TM275 - Parallel Programming with Python

Parallel Programming with
Python
• multiprocessing includes a very simple and intuitive
API to divide work between multiple processes.
import multiprocessing

#verify the number of CPUs available

multiprocessing.cpu_count()

TM275 - Parallel Programming with Python

16
Parallel Programming with
Python

Create the Wait till the

Definer functions different Run Processes completion of
processes processes

TM275 - Parallel Programming with Python

17
Parallel Programming with
Python
• Create Process

p = multiprocessing.Process(target=function_name,
args=(10, ))
• target: function name to be executed by the process
• args: arguments to be passed to the function
Create Wait till the
Define Run
different completion of
functions processes processes
processes

TM275 - Parallel Programming with Python

18
Parallel Programming with
Python
• Run the process
p.start()

• Once the processes are started, the current program also continues to run.
• To stop running the current program until a process is completed:

• p.join()

Create Wait till the

Define Run
different completion of
functions processes processes
processes

TM275 - Parallel Programming with Python

19
Parallel Programming with
Python– Example 1
• Create a “print_cube” function that prints the cube of a
number on the screen.
• Create a “print_carre” function which prints the square
of a number on the screen.
• Run 2 functions in parallel Show “Done! » once the 2
functions finish.

TM275 - Parallel Programming with Python

20
 Parallel Programming with
Python – Example 1
if __name__ == '__main__':
# import the multiprocessing
module # creer les processus
import multiprocessing p1 = multiprocessing.Process(target=print_square,
args=(10, ))
def print_cube(num):
p2 = multiprocessing.Process(target=print_cube,
"""
args=(10, ))
function that prints on the screen the cube of a number
""" # run process 1
print(Cube« S: {}".format(num * num * num)) p1.start()
# run process 2
p2.start()

def print_square(num): # wait until process 1 is finished

"""
p1.join()
function that prints on the screen the square of a number# wait until process 2 is finished
""" p2.join()
print("Square: {}".format(num * num))
# both processes finished
TM275 - Parallel Programming with Python
print("Done!")
21
Parallel Programming with
Python– Example 2
import multiprocessing as mp if __name__ == "__main__":
p1 = mp.Process(target = print_100, args=(1,))
p2 = mp.Process(target = print_100, args=(2,))
def print_100(id): p3 = mp.Process(target = print_100, args=(3,))
for i in range(100):
print("I am in processus{}".format(id)) p1.start()
p2.start()
p3.start()

p1.join()
p2.join()
p3.join()

print("Done!")

TM275 - Parallel Programming with Python

22
Parallel Programming with
Python
• Get Process ID
• os.getpid()
• Process_name.pid

• Verify if process is alive

• Process_name.is_alive()

TM275 - Parallel Programming with Python

23
Parallel Programming with
Python– Exercise
• Create the functions “worker 1” and “worker 2” which
print the pid of their process on the screen.
• Use the os.getpid() function
• Run 2 functions in parallel
• Display the IDs of the parent process and the 2 child
processes outside of functions
• Show “Done! » once the 2 functions finish.

TM275 - Parallel Programming with Python

24
 Parallel Programming with
Python - Example 3
import multiprocessing

# empty list with global scope if __name__ == "__main__":

result = [] # input list
mylist = [1,2,3,4]
def square_list(mylist):
""" # creating new process
function to square a given list p1 = multiprocessing.Process(target=square_list
""" args=(mylist,))
global result # starting process
# append squares of mylist to global p1.start()
list result # wait until process is finished
for num in mylist: p1.join()
result.append(num * num)
# print global list result # print global result list
print("Result(in process p1): print("Result(in main program):
{}".format(result)) {}".format(result))
TM275 - Parallel Programming with Python

25
Programmation Parallèle avec
Python - Example 3

Result(in process p1): [1, 4, 9, 16]

Result(in main program): [ ]

TM275 - Parallel Programming with Python

26
Sharing Memory between Processes
• multiprocessing.Shared Memory

TM275 - Parallel Programming with Python

27
Sharing Memory between Processes
• multiprocessing.Shared Memory
• provides Array and Value objects to share data between
processes
• Array: a ctypes array allocated from shared memory.
• Value: a ctypes object allocated from shared memory.

TM275 - Parallel Programming with Python

28
Sharing Memory between Processes
– Shared Memory
1. Create an array or avaleur
ar1 = multiprocessing.Array(data_type, array_size)
‘i’: int
‘d’ : float

val1 = multiprocessing.Value( data_type)

val1 = multiprocessing.Value( data_type, vinitial_value)

TM275 - Parallel Programming with Python

29
Sharing Memory between Processes
– Shared Memory
2. Pass ar1 and val1 as arguments while creating
processes

p1 = multiprocessing.Process(target=function_name,
args=(argument, ar1 , val1))

TM275 - Parallel Programming with Python

30
Sharing Memory between Processes
– Shared Memory
3. Acccess elements of ar1 and val1

ar1[indice]
val1.value

TM275 - Parallel Programming with Python

31
 Sharing Memory between Processes
– Shared Memory – Example 4
• Adopting Example 3 if __name__ == "__main__":
# input list
from multiprocessing import Process, Array, value mylist = [1,2,3,4]

def square_list(mylist): #creer un array

""" ar1 = Array('i',4)
function to square a given list
""" # creating new process
indice = 0 p1 = multiprocessing.Process(target=square_list,
# append squares of mylist to global list result args=(mylist,ar1))
for num in mylist: # starting process
ar1[indice] = (num * num) p1.start()
indice+=1 # wait until process is finished
p1.join()
# print list
print("Result(in process p1): , ar1[:]) # print global result
TM275 list
- Parallel Programming with Python
print("Result(in main program): {}“, ar1[:]) 32
 Sharing Memory between Processes
- Server Process
Chaque fois qu'un programme python démarre, un
processus serveur est également lancé.

À partir de là, chaque Demande d’un

fois on a besoin d’un processus fils Processus Serveur
Processus
nouveau processus, le fils
processus parent se Processus
connecte au serveur Père
et lui demande de
créer un nouveau Processus
processus. fils

Le module multiprocessing fournit une classe Manager qui contrôle un processus serveur..
Un processus serveur peut contenir des objets Python et permettre à d'autres processus de les manipuler à l'aide de proxys.
TM275 - Parallel Programming with Python
Par conséquent, les Managers fournissent un moyen de créer des données qui peuvent être partagées
33
entre différents processus.
Partage de Mémoire entre les
processus- Server Process
1- Créer un objet manager

manager = multiprocessing.Manager()

TM275 - Parallel Programming with Python

34
Sharing Memory between Processes
– Server Process
2 – créer une liste dans la memoire du server process
server_list = manager.list([])

2- créer un dictionnaire dans la memoire du server

process
server_dict = manager.dict({})

TM275 - Parallel Programming with Python

35
Sharing Memory between Processes
– Server Process
3- Passer server_list/server_dict dans les arguments du
processus

P1 = multiprocessing.Process(target = nom_function,
args =(argument_function, server_list)

TM275 - Parallel Programming with Python

36
Sharing Memory between Processes
– Server Process - Example

Processus Serveur Processus fils p1

Pour insérer un
Processus nouveau record
Père
Records= [ (‘Sam’,10), Processus fils p2
(‘Adam’, 9), pour imprimer le
(‘Kevin’, 9)] contenu de recors

(Nom, Score)

TM275 - Parallel Programming with Python

37
Sharing Memory between Processes
– Server Process - Example
import multiprocessing

def print_records(records):
"""
function to print element(tuples) dans records(list)
"""
for element in records:
print("Nom: {0}\nScore: {1}\n".format(element[0], element[1]))

def ajouter_element(element, records):

"""
fonction pour ajouter un nouvel element dans records(list)
"""
records.append(element)
print(“Nouvel element ajoute!\n")
TM275 - Parallel Programming with Python

38
 Sharing Memory between Processes
– Server Process - Example
# creating new processes
p1 = multiprocessing.Process(target=ajouter_element,
if __name__ == '__main__': args=(new_record, records))
#with multiprocessing.Manager() as manager: p2 = multiprocessing.Process(target=print_records,
manager = multiprocessing.Manager() args=(records,))

# creating a list in server process memory

records = manager.list([('Sam', 10), ('Adam', 9), ('Kevin',9)]) # running process p1 to insert new record
p1.start()
# new record to be inserted in records p1.join()
new_record = ('Jeff', 8)
# running process p2 to print records
p2.start()
p2.join()

TM275 - Parallel Programming with Python

39
Communication entre les
processus
• L'utilisation efficace de plusieurs processus nécessite généralement une
certaine communication entre eux, de sorte que le travail peut être divisé et
les résultats peuvent être agrégés.
• « multiprocessing » prend en charge deux types de canaux de communication
entre processus :

Queue Pipe

TM275 - Parallel Programming with Python

40
Communication entre les
processus - Queue
• Creer une multiplocessing Queue
• q = multiprocessing.Queue()
• Mettre dans la queue
• q.put()
q.put() q.get()
16 9 4 1 2
• Lire de la queue
• q.get()
• Verifier si la queue est vide
• q.empty()

TM275 - Parallel Programming with Python

41
Communication entre les
processus – Queue - Example

Ajouter un nouvel élément

à la queue

Imprimer le contenu de la
queue TM275 - Parallel Programming with Python

42
Communication entre les
processus – Queue - Example
import multiprocessing if __name__ == "__main__":
# input list
def square_list(mylist, q): mylist = [1,2,3,4]
"""
function to square a given list # creating multiprocessing Queue
""" q = multiprocessing.Queue()
# append squares of mylist to queue # creating new processes
for num in mylist: p1 = multiprocessing.Process(target=square_list, args=(mylist, q))
q.put(num * num) p2 = multiprocessing.Process(target=print_queue, args=(q,))
def print_queue(q):
""" # running process p1 to square list
function to print queue elements p1.start()
""" p1.join()
print("Queue elements:")
# running process p2 to get queue elements
while not q.empty():
p2.start() TM275 - Parallel Programming with Python
print(q.get())
p2.join()
print("Queue is now empty!") 43
Communication entre les
processus - Pipes
• Créer une Pipe
• parent_conn, child_conn = multiprocessing.Pipe()
• Retourne 2 objects représentant les 2 points de
communication de la pipe
• Envoyer un message
• Conn.send(message)
Conn 1 9 4 1 Conn
• Recevoir le message A B
• message = conn.recv()

TM275 - Parallel Programming with Python

44
Communication entre les
processus – Pipes - Example

Ajouter les éléments d’une liste dans la pipe

Imprimer les éléments reçus de l’autre côté

de la pipe

TM275 - Parallel Programming with Python

45
Communication entre les
processus – Pipes - Example
import multiprocessing

def ajouter_elements(my_list, parent_conn):

"""
Fonction pour envoyer les elements dans la pipe
"""
for element in my_list:
parent_conn.send(element)
print("{} envoyé".format(element))

def imprimer_elements_recus(child_conn):
""" Fonction pour imprimer les elements reçus de la pipe """
while True:
msg = child_conn.recv()
if (msg=='END'):
print("Fin!") TM275 - Parallel Programming with Python

break 46
print("{} reçu".format(msg))
 Communication entre les
processus – Pipes - Example
if __name__ == "__main__":

#creer et initialiser une liste

my_list = ["hello","hey","hru?",'END']

#creer une pipe

parent_conn, child_conn = multiprocessing.Pipe()
#creer les 2 processus
p1 = multiprocessing.Process(target = ajouter_elements, args = (my_list, parent_conn))
p2 = multiprocessing.Process(target = imprimer_elements_recus, args = (child_conn,))
#Run processes
p1.start()
p2.start()
#attendre la fin des processus
p1.join() TM275 - Parallel Programming with Python
p2.join()
47
print("Done!")
Processus Concurrents
• 2 processus concurrents exécutent simultanément un
segment de programme particulier appelé section
critique.
• La section critique fait référence aux parties du
programme où la ressource partagée est accessible.

TM275 - Parallel Programming with Python

48
Processus Concurrents
• Des accès simultanés à des ressources partagées peuvent entraîner des
situations de concurrence.
• Une concurrence se produit lorsque deux processus ou plus peuvent accéder à
des données partagées et tentent de les modifier en même temps.
• Par conséquent, les valeurs des variables peuvent être imprévisibles et varier
en fonction des horaires des changements de contexte des processus.

TM275 - Parallel Programming with Python

49
 Processus Concurrents – Example
def perform_transactions():
# Python program to illustrate
# the concept of race condition # balance initiale (dans shared memory)
# in multiprocessing balance = multiprocessing.Value('i', 100)
import multiprocessing
# creer nouveaux processus
# fonction pour le retrait de compte p1 = multiprocessing.Process(target=retrait, args=(balance,))
def retrait(balance): p2 = multiprocessing.Process(target=depot, args=(balance,))
for _ in range(10000):
balance.value = balance.value - 1 # starting processes
p1.start()
p2.start()
# fonction pour le depot dans compte # wait until processes are finished
def depot(balance): p1.join()
for _ in range(10000): p2.join()
balance.value = balance.value + 1 TM275 - Parallel Programming with Python

# print final balance 50

print("Balance Finale = {}".format(balance.value))
Processus Concurrents– Example
if __name__ == "__main__": 10 000 transactions de retrait et 10 000 transactions de
for _ in range(10): dépôt sont effectuées avec un solde initial de 100. Le solde
final attendu est de 100, mais ce que nous obtenons en 10
# perform same transaction process 10 times itérations de la fonction perform_transactions correspond
perform_transactions() à des valeurs différentes.
Cela se produit en raison de l'accès simultané des
processus des données partagées. Cette imprévisibilité de
Balance Finale = 2687 la valeur du solde n'est rien d'autre qu'une race condition.
Balance Finale = -249
Balance Finale = -2363
Balance Finale = 2551
Balance Finale = 1015
Balance Finale = -1121
Balance Finale = 1905
Balance Finale = 100
Balance Finale = -140 TM275 - Parallel Programming with Python
Balance Finale = -1270
51
Synchronisation entre les Processus
avec Python
• Mécanisme qui garantit que deux ou plusieurs
processus concurrents n'exécutent pas simultanément
un segment de programme particulier appelé section
critique.

TM275 - Parallel Programming with Python

52
Synchronisation entre les Processus
avec Python - Locks
Créer
• lock = multiprocessing.Lock()
un
Lock
Passe
r le
lock
en
argu
• p1 = multiprocessing.Process(target=withdraw, args=(balance,lock))
ment
Appli
de
quer la
functi
le
on
lock
target
dans
la
• lock.acquire()
sectio • Dès qu'un verrou est acquis, aucun autre processus ne peut accéder à sa section critique tant que le verrou n'est pas libéré
n
critiq
ue de
la
functi
on
Libere
target
• lock.release()
r le
Lock

TM275 - Parallel Programming with Python

53
Pooling des Processus avec Python

# Python program to find

# squares of numbers in a given list
def square(n):
return (n*n)

if __name__ == "__main__":

# input list
mylist = [1,2,3,4,5]

# empty list to store result

result = []

for num in mylist:

result.append(square(num))

print(result)

Un seul des cores est utilisé pour Utiliser les différents cores
[1, 4, 9, 16, 25] l'exécution du programme et il est tout à
fait possible que d'autres coers restent
inactifs. TM275 - Parallel Programming with Python

54
Pooling des Processus avec Python
• La tâche est déchargée/distribuée automatiquement entre les
cœurs/processus par l'objet Pool.
• L'utilisateur n'a pas à se soucier de la création explicite de processus.

TM275 - Parallel Programming with Python

utilise tous les cores
55
Pooling des Processus avec Python
1- p = multiprocessing.Pool()
• processes: spécifie le nombre de processus.
• maxtasksperchild: spécifie le nombre maximal de taches
attribué à chaque processus fils.
• Tous les processus d'un pool peuvent être amenés à effectuer
une initialisation à l'aide de ces arguments :
• initializer: spécifie une fonction d’initialization
• initargs: arguments à passer à l'initialiseur

TM275 - Parallel Programming with Python

56
Pooling des Processus avec Python
2-mapper à une fonction.
• result = p.map(nom_fonction, mylist)

TM275 - Parallel Programming with Python

57
Pooling des Processus avec Python –
Example
import multiprocessing

def square(n):
return n*n
if __name__ == "__main__":

#liste d'entree
my_list = [1,2,3,4]

#creer pool
pool = multiprocessing.Pool()
# map list to target function
result = pool.map(square, my_list)

print(result) TM275 - Parallel Programming with Python

58
Merci
End of Meeting 2
Thank you
TM275 - Parallel Programming with Python

61