How to use for loop with dask for parallel processing?
This recipe helps you use for loop with dask for parallel processing
Recipe Objective
How to use for loop with dask for parallel processing.
We will transform the function inc to be used parallely with the help of dask delayed, and we will show you the visualizaion of the for loop to understand parallel working of it.
Table of Contents
Step 1- Importing Libraries.
from dask import delayed
Step 2- Defining a function.
We will define a incremental function which will increase the value by 2 in a for loop.
def inc(x):
return x + 2
Step 3- Applying delayed.
We will apply dask delayed in the for loop to run inc function parallely.
data = [1, 2, 3, 4, 5]
results = []
for x in data:
y = delayed(inc)(x)
results.append(y)
Step 4- Displaying results.
We will print the results of calculating addition and final computing.
addition = delayed(sum)(results)
print("Before addition:", addition)
calculate = addition.compute()
print("After calculating :", calculate)
Step 5- Visualizing
Visualizing the whole addition process that was done in parallel.
addition.visualize()
What Users are saying..
Ed Godalle
I am the Director of Data Analytics with over 10+ years of IT experience. I have a background in SQL, Python, and Big Data working with Accenture, IBM, and Infosys. I am looking to enhance my skills... Read More