Skip to main content

MultiProcessing in Python

 


Learning Sections          show

MultiProcessing in Python

Multiprocessing in Python involves using the multiprocessing module to run multiple processes concurrently, taking advantage of multiple CPU cores. This module provides a higher level of concurrency than threading and is especially useful for CPU-bound tasks.

Creating Processes

You can create and start a new process by using the multiprocessing module:


import multiprocessing

def print_numbers():
    for i in range(10):
        print(i)

p1 = multiprocessing.Process(target=print_numbers)
p1.start()
p1.join()  # Wait for the process to complete
Using Process Pools

The multiprocessing module provides a Pool class, which allows you to manage a pool of worker processes:


from multiprocessing import Pool

def square(n):
    return n * n

with Pool(4) as pool:
    result = pool.map(square, range(10))
    print(result)
Inter-Process Communication

Processes can communicate using pipes or queues. The multiprocessing.Queue class provides a FIFO mechanism for this purpose:


import multiprocessing

def producer(q):
    for i in range(5):
        q.put(i)
        print("Produced", i)

def consumer(q):
    while not q.empty():
        item = q.get()
        print("Consumed", item)

q = multiprocessing.Queue()
p1 = multiprocessing.Process(target=producer, args=(q,))
p2 = multiprocessing.Process(target=consumer, args=(q,))

p1.start()
p2.start()
p1.join()
p2.join()
Shared Memory

The multiprocessing module provides shared memory capabilities using Value or Array:


import multiprocessing

def increment(shared_value):
    with shared_value.get_lock():  # Synchronize access
        shared_value.value += 1

shared_value = multiprocessing.Value('i', 0)
processes = [multiprocessing.Process(target=increment, args=(shared_value,)) for _ in range(10)]

for p in processes:
    p.start()

for p in processes:
    p.join()

print(shared_value.value)  # Should print 10
Daemon Processes

Like threads, processes can also be run as daemons. Daemon processes are terminated when the main program exits:


import multiprocessing
import time

def background_task():
    while True:
        print("Running in the background")
        time.sleep(2)

p = multiprocessing.Process(target=background_task)
p.daemon = True
p.start()

print("Main process exiting")
Labels:

Popular posts from this blog

Generators in Python

  Learning Sections          show Generators in Python Generators are a special type of iterator in Python that allow you to iterate over a sequence of items without storing them all in memory at once. They are useful for generating large sequences of data on-the-fly, or for processing data in a memory-efficient manner. Creating Generators In Python, generators are created using generator functions or generator expressions: # Generator function def my_generator ( n ): for i in range ( n ): yield i # Generator expression my_generator = ( i for i in range ( 10 )) A generator function uses the yield keyword to yield values one at a time, while a generator expression creates an anonymous generator. Iterating Over Generators You can iterate over the values produced by a generator using a for loop: for value in my_generator ( 5 ): print ( value ) This w...
Read more

Inheritance in Python

  Learning Sections          show Inheritance in Python Inheritance is a fundamental concept in object-oriented programming (OOP) that allows a class to inherit attributes and methods from another class. The class that inherits is called the child class or subclass, and the class being inherited from is called the parent class or superclass. Basic Inheritance In Python, a child class inherits from a parent class by specifying the parent class in parentheses after the child class name. Example: class Animal : def __init__ ( self , name ): self . name = name def speak ( self ): raise NotImplementedError ( "Subclass must implement this method" ) class Dog ( Animal ): def speak ( self ): return "Woof!" class Cat ( Animal ): def speak ( self ): return "Meow!" # Create instances of Dog and Cat dog = Dog ( "Buddy" ) cat = Cat ( "Whiskers" ...
Read more

If else Conditional Statements in Python

  Learning Sections     show If-Else Conditional Statements Conditional statements allow you to execute different blocks of code based on certain conditions. The most common conditional statement is the if statement. It can be used alone, or combined with elif (else if) and else statements to handle multiple conditions. If Statement The if statement evaluates a condition, and if the condition is true, the block of code indented under the if statement is executed. # If statement example x = 10 if x >> 0 : print ( "x is positive" ) If-Else Statement The if-else statement adds an additional block of code that runs if the condition is false. # If-else statement example x = -10 if x >> 0 : print ( "x is positive" ) else : print ( "x is non-positive" ) If-Elif-Else Statement The if-elif-else statement allows you to check multiple conditions. The fir...
Read more