Automation & Analytics using Python

|Year: 2023-24

Chapter-1

INTRODUCTION
In today's fast-paced digital world, automation and data analytics have become critical
components of many industries. Automation refers to the use of technology to perform tasks
with minimal human intervention, enhancing efficiency and accuracy. Data analytics involves
examining datasets to extract meaningful insights and support decision-making processes.
Both automation and analytics have widespread applications, ranging from business
operations and financial services to healthcare and marketing.

1.1 The Role of Python in Automation and Analytics

Python has emerged as a leading programming language for both automation and analytics
due to its simplicity, versatility, and extensive ecosystem of libraries and tools. Python's clean
and readable syntax makes it accessible for beginners, while its powerful capabilities meet
the needs of experienced developers and data scientists.

1.2 Why Python?

Several factors contribute to Python's popularity in these fields:

 Extensive Libraries and Frameworks: Python offers a rich collection of libraries

for automation (e.g., Selenium, BeautifulSoup, PyAutoGUI) and data analytics (e.g.,
Pandas, NumPy, Matplotlib, Scikit-Learn).

 Ease of Learning and Use: Python's syntax is straightforward and easy to learn,
which reduces the learning curve and allows for rapid development and prototyping.
 Community and Support: Python has a large, active community that continuously
contributes to its development, providing a wealth of resources, tutorials, and support.

 Cross-Platform Compatibility: Python runs on various operating systems, including

Windows, macOS, and Linux, making it a versatile choice for different environments.

1.3 Importance of Automation

Automation streamlines repetitive tasks, reduces errors, and frees up human resources for
more strategic activities. In industries like manufacturing, finance, and IT, automation is used
to perform tasks such as data entry, report generation, web scraping, and software testing. By

Department of CS&E,SJMIT,Chitradurga Page 1

 Automation & Analytics using Python
|Year: 2023-24

leveraging Python for automation, organizations can improve operational efficiency, ensure
consistency, and respond more swiftly to business needs.

1.4 Importance of Analytics

Data analytics transforms raw data into valuable insights, enabling organizations to make
informed decisions. Python's powerful data analysis libraries allow users to manipulate,
visualize, and model data effectively. In sectors like healthcare, marketing, and finance, data
analytics helps uncover trends, predict outcomes, and optimize strategies. Python's
capabilities in machine learning and statistical analysis further enhance its utility in deriving
actionable insights from data.

1.5 Python in Practice

This report explores the key aspects of automation and analytics using Python. It covers
essential libraries, practical examples, advanced techniques, real-world applications, best
practices, and challenges. By understanding these concepts, professionals can harness
Python's full potential to automate tasks and analyze data efficiently, ultimately driving
innovation and success in their respective fields.

In the following sections, we will delve into the specifics of using Python for automation and
analytics, providing a comprehensive guide to leveraging this powerful toolset in real-world
scenarios.

Department of CS&E,SJMIT,Chitradurga Page 2

 Automation & Analytics using Python
|Year: 2023-24

Chapter-2

PYTHON
Python is a high-level, interpreted programming language known for its simplicity,
versatility, and readability. Guido van Rossum created Python in the late 1980s, and it has
since become one of the most popular programming languages worldwide. Python's design
philosophy emphasizes code readability, with its clear and concise syntax making it
accessible to both beginners and experienced developers alike.

2.1 Key Features:

 Simple and Readable Syntax: Python's syntax is clean, straightforward, and easy to
understand, making it ideal for beginners and facilitating rapid development.

 Interpreted and Interactive: Python is an interpreted language, meaning that code is

executed line by line, which allows for rapid prototyping and debugging. It also
supports interactive mode, enabling users to execute code interactively in a REPL
(Read-Eval-Print Loop) environment.

 Dynamic Typing and Automatic Memory Management: Python uses dynamic

typing, where variable types are inferred at runtime, making it flexible and adaptable.
It also features automatic memory management, handling memory allocation and
deallocation transparently to the programmer.

 Extensive Standard Library: Python comes with a comprehensive standard library

that provides built-in modules and functions for various tasks, including file I/O,
networking, data compression, and more. This extensive library ecosystem reduces
the need for external dependencies and accelerates development.

 Cross-Platform Compatibility: Python is platform-independent, meaning that code

written in Python can run seamlessly on different operating systems, including
Windows, macOS, and Linux, without modification.

Department of CS&E,SJMIT,Chitradurga Page 3

 Automation & Analytics using Python
|Year: 2023-24

 High-level Data Structures: Python provides built-in support for high-level data
structures such as lists, tuples, dictionaries, and sets, making it well-suited for tasks
involving data manipulation and analysis.

 Object-Oriented and Functional Programming: Python supports both object-

oriented and functional programming paradigms, allowing developers to write
modular, reusable, and maintainable code. It also provides support for features like
inheritance, polymorphism, and encapsulation.

2.2 Applications:
Python's versatility makes it suitable for a wide range of applications, including:

 Web Development: Frameworks like Django and Flask enable rapid development of
web applications.

 Data Science and Analytics: Libraries like NumPy, Pandas, and Matplotlib support
data manipulation, analysis, and visualization.

 Machine Learning and Artificial Intelligence: Libraries like Scikit-Learn,

TensorFlow, and PyTorch provide tools for building and training machine learning
models.

 Scripting and Automation: Python is commonly used for scripting tasks,

automation, and system administration.

 Game Development: Libraries like Pygame support game development, including

graphics, audio, and input handling.

 Desktop GUI Applications: Libraries like Tkinter and PyQt allow developers to
create cross-platform desktop GUI applications.

 Community and Ecosystem:Python has a large and active community of developers,

enthusiasts, and contributors who continually contribute to its growth and
improvement. The Python Package Index (PyPI) hosts thousands of third-party
packages and libraries, providing additional functionality and extending Python's
capabilities in various domains.
Department of CS&E,SJMIT,Chitradurga Page 4
 Automation & Analytics using Python
|Year: 2023-24

Chapter-3

AUTOMATION USING PYTHON

3.1 Definition and Purpose of Automation

Automation refers to the use of technology to perform tasks with minimal human
intervention. Its primary purpose is to increase efficiency, accuracy, and speed in executing
repetitive or complex processes. By automating mundane tasks, organizations can redirect
human effort toward more strategic and creative activities, thereby enhancing overall
productivity and innovation.

3.2 Historical Context and Evolution

The concept of automation is not new; it has evolved significantly over time. The industrial
revolution introduced mechanical automation in manufacturing, dramatically improving
production capabilities. In the mid-20th century, the advent of computers paved the way for
digital automation, enabling more complex and precise control over various processes.
Today, with advancements in artificial intelligence and machine learning, automation has
reached new heights, allowing for intelligent decision-making and adaptive systems.

3.3 Types of Automation

Automation can be broadly categorized into several types, each serving different purposes
and application areas:

 Industrial Automation: Involves the use of control systems, such as computers or

robots, to handle industrial processes and machinery. Examples include assembly line
robots, CNC machines, and automated quality control systems.

 Office Automation: Focuses on streamlining office tasks, such as data entry,

scheduling, and document management. Tools like spreadsheets, email clients, and
word processors fall under this category.

Department of CS&E,SJMIT,Chitradurga Page 5

 Automation & Analytics using Python
|Year: 2023-24

 Business Process Automation (BPA): Automates complex business processes and

workflows. Examples include customer relationship management (CRM) systems,
enterprise resource planning (ERP) systems, and automated invoicing systems.

 IT Automation: Involves the use of software and tools to automate IT infrastructure

and operations. Examples include automated software deployment, network
management, and server monitoring.

 Home Automation: Refers to the automation of household activities, such as

lighting, heating, and security systems. Smart home devices like thermostats, security
cameras, and voice assistants are examples.

3.4 Key Benefits of Automation:

 Increased Efficiency: Automation can perform tasks faster and more accurately than
humans, leading to significant time savings and increased output.

 Cost Reduction: By reducing the need for manual labor and minimizing errors,
automation can lower operational costs.

 Improved Accuracy and Consistency: Automated systems are less prone to human
error, ensuring consistent and accurate results.

 Scalability: Automation allows processes to scale effortlessly, handling large

volumes of work without the need for additional resources.

 Enhanced Productivity: Freeing employees from repetitive tasks enables them to

focus on higher-value activities, boosting overall productivity and job satisfaction.

3.5 Challenges and Considerations:

While automation offers numerous benefits, it also presents several challenges:

 Initial Investment: Implementing automation systems can require significant upfront

costs in terms of technology, infrastructure, and training.

Department of CS&E,SJMIT,Chitradurga Page 6

 Automation & Analytics using Python
|Year: 2023-24

 Complexity: Designing and maintaining automated systems can be complex,

requiring specialized skills and expertise.

 Job Displacement: Automation can lead to job displacement as machines replace

human roles. Organizations must manage this transition carefully to minimize
negative impacts on the workforce.

 Security Risks: Automated systems can be vulnerable to cyber-attacks and data

breaches, necessitating robust security measures.

 Adaptability: Automated systems may struggle to adapt to unexpected changes or

unique scenarios, requiring human oversight and intervention.

3.6 The Role of Python in Automation

Python has become a popular choice for automation due to its simplicity, versatility, and
extensive ecosystem of libraries. Python's capabilities in automation span across various
domains, including:

 Web Scraping and Data Extraction: Libraries like BeautifulSoup and Scrapy allow
for efficient extraction of data from websites.

 Browser Automation: Selenium enables the automation of web browser interactions,

useful for testing and data collection.

 Task Scheduling: Python's Schedule library provides simple and flexible task
scheduling capabilities.

 GUI Automation: PyAutoGUI allows for the automation of keyboard and mouse
actions, enabling control over graphical user interfaces.

 API Integration: The Requests library facilitates interaction with web APIs,
automating data exchange and service interactions.

 Testing Automation: Pytest and other testing frameworks automate software testing
processes, ensuring reliable and robust applications.

Department of CS&E,SJMIT,Chitradurga Page 7

 Automation & Analytics using Python
|Year: 2023-24

3.7 Key Libraries for Automation

Automation in Python is facilitated by a variety of libraries and tools that streamline

repetitive tasks, interact with web browsers, parse HTML content, and more. Two key
libraries for automation are Selenium and BeautifulSoup.

3.7.1 Selenium

Overview:

Selenium is a powerful tool for automating web browsers. It provides a WebDriver API that
allows you to interact with web elements, simulate user actions, and execute JavaScript
within the browser. Selenium supports multiple programming languages, including Python,
Java, C#, and JavaScript.

Features:

 Cross-Browser Compatibility: Selenium supports automation across different web

browsers such as Chrome, Firefox, Safari, and Internet Explorer.

 Element Identification: Selenium enables you to locate and interact with HTML
elements using various locators such as ID, class name, CSS selector, XPath, etc.

 User Actions Simulation: You can simulate user interactions like clicking buttons,
filling forms, scrolling, and hovering over elements.

 JavaScript Execution: Selenium allows executing JavaScript code within the

browser, enabling advanced interactions and manipulations.

 Headless Browser Support: Selenium supports headless browser automation,

allowing you to run browser automation without a graphical interface.

 Testing Framework Integration: Selenium can be integrated with testing

frameworks like Pytest and unittest for automated testing of web applications.

Example:

Department of CS&E,SJMIT,Chitradurga Page 8

 Automation & Analytics using Python
|Year: 2023-24

from selenium import webdriver

# Launch a Chrome browser instance

driver = webdriver.Chrome()

# Open a webpage
driver.get('https://example.com')

# Find and interact with elements

element = driver.find_element_by_id('some_id')
element.click()

# Close the browser

driver.quit()

3.7.2 BeautifulSoup

BeautifulSoup is a Python library for parsing HTML and XML documents, extracting data,
and navigating the parse tree. It simplifies the process of web scraping by providing easy-to-
use methods for locating and extracting specific elements from web pages.

Features:

 HTML Parsing: BeautifulSoup parses HTML documents and constructs a parse tree,
making it easy to navigate and extract data.

 Element Extraction: You can extract data from HTML elements based on attributes,
tags, classes, and more.

 Data Extraction: BeautifulSoup provides methods for extracting text, attributes, and
other data from HTML elements.

 Navigating the Parse Tree: You can navigate the HTML parse tree using methods
like find, find_all, children, parent, siblings, etc.

Department of CS&E,SJMIT,Chitradurga Page 9

 Automation & Analytics using Python
|Year: 2023-24

 Integration with Requests: BeautifulSoup is often used in conjunction with the

Requests library for fetching web pages and parsing their content.

Example:
import requests
from bs4 import BeautifulSoup

# Fetch a webpage
response = requests.get('https://example.com')
html_content = response.text

# Parse the HTML content

soup = BeautifulSoup(html_content, 'html.parser')

# Extract data from HTML elements

title = soup.title.text
paragraphs = soup.find_all('p')

for p in paragraphs:
print(p.text)

These two libraries, Selenium and BeautifulSoup, are essential tools for automating web
interactions, scraping web content, and extracting data from HTML documents in Python.
They empower developers to automate tasks such as web scraping, testing, and browser
automation with ease and efficiency.

Department of CS&E,SJMIT,Chitradurga Page 10

 Automation & Analytics using Python
|Year: 2023-24

Chapter-4

ANALYTICS USING PYTHON

Data analytics is the process of examining large datasets to uncover patterns, trends,
correlations, and other insights that can inform decision-making and drive business strategies.
It involves various techniques, tools, and methodologies for extracting valuable information
from raw data, which can be structured or unstructured. Data analytics plays a crucial role in
diverse fields such as business, finance, healthcare, marketing, and scientific research,
enabling organizations to gain a competitive edge, optimize operations, and innovate.

4.1 Key Components of Data Analytics

 Data Collection: The first step in data analytics involves gathering data from multiple
sources, including databases, files, sensors, APIs, social media, and IoT devices. Data
can be structured (e.g., databases, spreadsheets) or unstructured (e.g., text, images,
videos).

 Data Preparation: Once collected, raw data often requires preprocessing and
cleaning to remove inconsistencies, missing values, duplicates, and outliers. Data
preparation tasks may also involve data transformation, normalization, and feature
engineering to make the data suitable for analysis.

 Exploratory Data Analysis (EDA): EDA involves visualizing and summarizing the
characteristics of the data to gain insights and identify patterns. Techniques such as

Department of CS&E,SJMIT,Chitradurga Page 11

 Automation & Analytics using Python
|Year: 2023-24

statistical summaries, data visualization (e.g., histograms, scatter plots, box plots), and
correlation analysis are commonly used in EDA.

 Statistical Analysis: Statistical analysis involves applying statistical methods to

analyze data and make inferences about underlying populations or relationships. It
includes descriptive statistics (e.g., mean, median, standard deviation), hypothesis
testing, regression analysis, and more.

 Machine Learning and Predictive Modeling: Machine learning techniques enable

the development of predictive models that can make predictions or classifications
based on historical data. Supervised learning, unsupervised learning, and
reinforcement learning are common types of machine learning algorithms used in data
analytics.

 Data Visualization: Data visualization is a crucial aspect of data analytics that

involves presenting data in graphical or visual formats to facilitate understanding and
interpretation. Visualization techniques include charts, graphs, heatmaps, dashboards,
and interactive visualizations.

4.2 Tools and Technologies

Several tools and technologies are used in data analytics to perform various tasks:

 Programming Languages: Python and R are widely used programming languages

for data analytics due to their rich ecosystem of libraries and tools (e.g., Pandas,
NumPy, Matplotlib, Scikit-Learn, TensorFlow).

 Data Visualization Tools: Tools like Tableau, Power BI, and matplotlib/seaborn in
Python are used for creating interactive visualizations and dashboards.

 Big Data Technologies: Technologies like Hadoop, Spark, and Apache Kafka are
used for processing and analyzing large volumes of data in distributed environments.
 Database Management Systems (DBMS): DBMS such as SQL Server, MySQL,
and PostgreSQL are used for storing and managing structured data, while NoSQL
databases like MongoDB and Cassandra are used for handling unstructured data.

Department of CS&E,SJMIT,Chitradurga Page 12

 Automation & Analytics using Python
|Year: 2023-24

4.3 Applications of Data Analytics

Data analytics has diverse applications across various industries:

 Business and Finance: Market analysis, customer segmentation, risk management,

fraud detection, and financial forecasting.

 Healthcare: Disease prediction, patient monitoring, personalized medicine, and drug

discovery.

 Marketing and Advertising: Customer profiling, campaign optimization, sentiment

analysis, and recommendation systems.

 Manufacturing and Supply Chain: Predictive maintenance, quality control, demand

forecasting, and supply chain optimization.

 Science and Research: Climate modeling, genomics, astrophysics, and social science
research.

4.4 Key Libraries for Data Analytics

Data analytics in Python is facilitated by a rich ecosystem of libraries and tools that offer
powerful capabilities for data manipulation, analysis, visualization, and modeling. Some of
the key libraries for data analytics in Python include:

4.4.1 Pandas

Pandas is a powerful Python library for data manipulation and analysis. It provides easy-to-
use data structures and functions for working with structured data, such as tabular data and
time series. Pandas is built on top of NumPy and is widely used in data science, finance,
research, and many other fields.

Key Features:

 DataFrame: Pandas introduces the DataFrame data structure, which is a two-

dimensional labeled data structure with columns of potentially different types. It
provides a flexible and efficient way to work with structured data.

Department of CS&E,SJMIT,Chitradurga Page 13

 Automation & Analytics using Python
|Year: 2023-24

 Series: Along with DataFrame, Pandas also provides the Series data structure, which
is a one-dimensional labeled array capable of holding any data type. Series are the
building blocks of DataFrame.

 Data Manipulation: Pandas offers a rich set of functions for data manipulation,
including indexing, slicing, filtering, grouping, merging, and reshaping data. These
functions allow for easy and intuitive data manipulation operations.

 Missing Data Handling: Pandas provides methods for handling missing or NaN (Not
a Number) values in data, including filling, dropping, and interpolating missing data.

 Data Alignment: Pandas automatically aligns data based on labels, making it easy to
perform operations on data with different indices or column names.

 Time Series Analysis: Pandas has built-in support for time series data, including
date/time indexing, resampling, and time zone handling. It makes working with time
series data intuitive and efficient.

 Input/Output: Pandas can read and write data from various file formats, including
CSV, Excel, JSON, SQL databases, and HDF5. It provides functions like read_csv(),
read_excel(), to_csv(), to_excel(), etc., for input/output operations.

 Data Visualization: While Pandas itself does not provide visualization capabilities, it
integrates well with other libraries like Matplotlib and Seaborn for data visualization.
It can easily generate plots and charts from DataFrame and Series data.

Example:

import pandas as pd

# Create a DataFrame from a dictionary

data = {'Name': ['Alice', 'Bob', 'Charlie', 'David'],

'Age': [25, 30, 35, 40],

'City': ['New York', 'Los Angeles', 'Chicago', 'Houston']}

Department of CS&E,SJMIT,Chitradurga Page 14

 Automation & Analytics using Python
|Year: 2023-24

df = pd.DataFrame(data)

# Display the DataFrame

print(df)

# Read data from a CSV file

data = pd.read_csv('data.csv')

Select a subset of data

subset = df[df['Age'] > 30]

# Group data by a column and compute statistics

grouped_data = df.groupby('City').mean()

# Plot data using Matplotlib

import matplotlib.pyplot as plt

df.plot(kind='bar', x='Name', y='Age', title='Age Distribution')

plt.show()

Pandas is an essential tool for data manipulation and analysis in Python. Its intuitive data
structures and functions make it easy to work with structured data, perform data manipulation
operations, and analyze data efficiently. Whether you're cleaning messy data, conducting
exploratory data analysis, or building predictive models, Pandas provides the tools you need
to work with data effectively.
Department of CS&E,SJMIT,Chitradurga Page 15
 Automation & Analytics using Python
|Year: 2023-24

4.4.2 NumPy

NumPy (Numerical Python) is a fundamental library for numerical computing in Python. It

provides support for multidimensional arrays, mathematical functions, linear algebra
operations, and random number generation. NumPy is widely used in scientific computing,
data analysis, machine learning, and many other fields.

Key Features:

 Arrays: NumPy introduces the ndarray (N-dimensional array) data structure, which is
a flexible container for homogeneous data. Arrays can have any number of
dimensions and can hold elements of any data type.
 Mathematical Functions: NumPy provides a wide range of mathematical functions
for performing element-wise operations on arrays. These functions include arithmetic
operations, trigonometric functions, exponential and logarithmic functions, and more.

 Linear Algebra: NumPy includes a comprehensive set of functions for linear algebra
operations, such as matrix multiplication, matrix inversion, eigenvalue decomposition,
singular value decomposition, and solving linear systems of equations.

 Random Number Generation: NumPy offers functions for generating random

numbers from various probability distributions, including uniform, normal
(Gaussian), binomial, and Poisson distributions. It also provides tools for shuffling
and sampling data.

 Indexing and Slicing: NumPy arrays support advanced indexing and slicing
operations, allowing you to extract subsets of data from arrays efficiently.

 Broadcasting: NumPy's broadcasting feature allows you to perform operations on

arrays of different shapes. It automatically aligns arrays based on their dimensions,
making it easier to write vectorized code.

 Integration with C/C++ and Fortran: NumPy is implemented in C and Python,

providing high performance and interoperability with other languages. It seamlessly
integrates with libraries written in C/C++ and Fortran for numerical computing.
Department of CS&E,SJMIT,Chitradurga Page 16
 Automation & Analytics using Python
|Year: 2023-24

Example

import numpy as np

# Create a 1D array

arr1d = np.array([1, 2, 3, 4, 5])

# Create a 2D array

arr2d = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Perform arithmetic operations on arrays

result = arr1d + 10

print(result)

# Perform linear algebra operations

matrix = np.array([[1, 2], [3, 4]])

inverse = np.linalg.inv(matrix)

print(inverse)

# Generate random numbers

random_numbers = np.random.rand(5)

print(random_numbers)

# Indexing and slicing

subset = arr2d[:, 1]

print(subset)

Department of CS&E,SJMIT,Chitradurga Page 17

 Automation & Analytics using Python
|Year: 2023-24

NumPy is an essential library for numerical computing in Python. Its powerful array data
structure and mathematical functions make it easy to perform complex numerical
computations efficiently. Whether you're working with large datasets, implementing machine
learning algorithms, or conducting scientific simulations, NumPy provides the tools you need
to work with numerical data effectively.

Department of CS&E,SJMIT,Chitradurga Page 18

 Automation & Analytics using Python
|Year: 2023-24

4.4.3 Matplotlib

Matplotlib is a comprehensive library for creating static, interactive, and publication-quality

visualizations in Python. It provides a wide range of plotting functions and customization
options for creating a variety of plots and charts, including line plots, scatter plots, bar charts,
histograms, heatmaps, and more. Matplotlib is widely used in scientific research, data
analysis, engineering, and many other fields.

Key Features

 Versatile Plotting: Matplotlib supports a wide range of plot types and styles,
allowing you to create almost any kind of plot imaginable. It provides functions for
creating line plots, scatter plots, bar charts, histograms, pie charts, box plots, violin
plots, heatmaps, and more.

 Customization: Matplotlib allows you to customize every aspect of your plots,

including colors, markers, line styles, labels, axes, titles, legends, and annotations. It
provides fine-grained control over plot appearance and layout, enabling you to create
visually appealing and informative plots.

 Multiple Backends: Matplotlib supports multiple backends for rendering plots,

including interactive backends for generating plots in interactive environments (e.g.,
Jupyter Notebooks) and non-interactive backends for generating plots in batch mode

Department of CS&E,SJMIT,Chitradurga Page 19

 Automation & Analytics using Python
|Year: 2023-24

(e.g., saving plots to files). This flexibility allows you to use Matplotlib in a variety of
workflows and environments.

 Integration with NumPy and Pandas: Matplotlib seamlessly integrates with NumPy
and Pandas, allowing you to create plots directly from NumPy arrays and Pandas
DataFrame objects. This makes it easy to visualize data stored in these data structures
and perform exploratory data analysis.

 Publication-Quality Output: Matplotlib produces high-quality plots suitable for

publication in scientific journals, reports, presentations, and other publications. It
provides options for saving plots in various file formats, including PNG, PDF, SVG,
EPS, and more.

 Support for LaTeX: Matplotlib supports LaTeX formatting for text elements in
plots, allowing you to use LaTeX syntax for mathematical expressions, symbols, and
fonts in plot labels, titles, annotations, and legends.

Example

import matplotlib.pyplot as plt

import numpy as np

# Create a simple line plot

x = np.linspace(0, 2*np.pi, 100)

y = np.sin(x)

plt.plot(x, y)

plt.xlabel('x')

plt.ylabel('sin(x)')

plt.title('Sine Function')

plt.grid(True)

plt.show()

Department of CS&E,SJMIT,Chitradurga Page 20

 Automation & Analytics using Python
|Year: 2023-24

# Create a scatter plot with custom markers and colors

x = np.random.rand(100)

y = np.random.rand(100)

sizes = np.random.rand(100) * 100

colors = np.random.rand(100)

plt.scatter(x, y, s=sizes, c=colors, alpha=0.5)

plt.xlabel('x')

plt.ylabel('y')

plt.title('Scatter Plot')

plt.colorbar(label='Color')

plt.show()

Department of CS&E,SJMIT,Chitradurga Page 21

 Automation & Analytics using Python
|Year: 2023-24

Department of CS&E,SJMIT,Chitradurga Page 22

 Automation & Analytics using Python
|Year: 2023-24

Department of CS&E,SJMIT,Chitradurga Page 23

 Automation & Analytics using Python
|Year: 2023-24

Matplotlib is an indispensable tool for data visualization and exploration in Python. Its
versatility, customization options, and publication-quality output make it suitable for a wide
range of plotting tasks, from simple exploratory data analysis to complex scientific
visualization. Whether you're visualizing data, presenting results, or creating publication-
quality plots, Matplotlib provides the tools you need to create informative and visually
appealing plots with ease.

4.4.4 Seaborn

Seaborn is a powerful Python library for creating attractive and informative statistical
graphics. Built on top of Matplotlib, Seaborn provides a high-level interface for creating
complex visualizations with minimal code. It offers a wide range of plotting functions and
customization options for creating various types of plots, including scatter plots, line plots,
bar plots, histograms, box plots, violin plots, heatmaps, pair plots, and more. Seaborn is
widely used in data analysis, statistical modeling, machine learning, and scientific research.

Department of CS&E,SJMIT,Chitradurga Page 24

 Automation & Analytics using Python
|Year: 2023-24

Key Features

 Statistical Visualization: Seaborn specializes in statistical visualization and provides

functions for visualizing relationships and distributions in data. It offers convenient
wrappers for common statistical plots and techniques, making it easy to create
informative visualizations.

 Integration with Pandas: Seaborn seamlessly integrates with Pandas DataFrame

objects, allowing you to create plots directly from DataFrame data. This makes it easy
to visualize data stored in Pandas DataFrames and perform exploratory data analysis.

 Attractive Aesthetics: Seaborn comes with built-in themes and styles that improve
the aesthetics of your plots and make them suitable for publication. It provides options
for customizing colors, fonts, grid lines, and other visual elements to create visually
appealing plots.

 Advanced Plot Customization: Seaborn provides extensive customization options

for fine-tuning the appearance and layout of your plots. It allows you to customize
plot elements such as colors, markers, line styles, axes, labels, titles, legends, and
annotations.

 Complex Plot Types: Seaborn supports a wide range of complex plot types and
techniques, including multi-plot grids, categorical plots, regression plots, time series
plots, distribution plots, and cluster maps. It provides functions for visualizing
relationships between multiple variables and identifying patterns in data.

 Integration with Matplotlib: Seaborn is built on top of Matplotlib and seamlessly

integrates with it. You can use Matplotlib functions alongside Seaborn functions to
create custom plots and combine multiple plots into complex visualizations.

Example

import seaborn as sns

import matplotlib.pyplot as plt

import pandas as pd
Department of CS&E,SJMIT,Chitradurga Page 25
 Automation & Analytics using Python
|Year: 2023-24

# Load sample dataset from Seaborn

tips = sns.load_dataset('tips')

# Create a scatter plot with regression line

sns.regplot(x='total_bill', y='tip', data=tips)

plt.xlabel('Total Bill')

plt.ylabel('Tip')

plt.title('Scatter Plot with Regression Line')

plt.show()

# Create a box plot

sns.boxplot(x='day', y='total_bill', data=tips)

plt.xlabel('Day of the Week')

plt.ylabel('Total Bill')

plt.title('Box Plot of Total Bill by Day of the Week')

plt.show()

# Create a pair plot

sns.pairplot(tips, hue='sex')

plt.show()

Department of CS&E,SJMIT,Chitradurga Page 26

 Automation & Analytics using Python
|Year: 2023-24

Seaborn is a versatile and powerful library for statistical visualization in Python. Its intuitive interface,
attractive aesthetics, and extensive customization options make it ideal for creating informative and
visually appealing plots for data analysis and exploration. Whether you're visualizing relationships,
distributions, or patterns in data, Seaborn provides the tools you need to create high-quality statistical
graphics with ease.

4.4.5 Scikit-Learn

Scikit-Learn is a comprehensive library for machine learning in Python. It provides simple

and efficient tools for data mining, data analysis, and predictive modeling. Scikit-Learn is
built on top of NumPy, SciPy, and Matplotlib, and it integrates seamlessly with these libraries
to provide a cohesive and powerful machine learning toolkit. Scikit-Learn is widely used in
academia, industry, and research for solving a wide range of machine learning tasks,
including classification, regression, clustering, dimensionality reduction, and model selection.

Key Features

 Unified Interface: Scikit-Learn provides a consistent and easy-to-use API across

different machine learning algorithms and techniques.this unified makes it easy to
experiment with different algorithms and compare their performance.

Department of CS&E,SJMIT,Chitradurga Page 27

 Automation & Analytics using Python
|Year: 2023-24

 Supervised Learning: Scikit-Learn supports various supervised learning algorithms

for classification and regression tasks. It includes algorithms such as Support Vector
Machines (SVM), Decision Trees, Random Forests, Gradient Boosting, k-Nearest
Neighbors (k-NN), and Neural Networks.

 Unsupervised Learning: Scikit-Learn provides algorithms for unsupervised learning

tasks such as clustering, dimensionality reduction, and density estimation. It includes
algorithms such as K-Means Clustering, Principal Component Analysis (PCA), t-
Distributed Stochastic Neighbor Embedding (t-SNE), and Gaussian Mixture Models
(GMM).

 Model Evaluation: Scikit-Learn includes functions and tools for evaluating the
performance of machine learning models using metrics such as accuracy, precision,
recall, F1 score, ROC AUC score, and mean squared error. It also provides functions
for cross-validation, grid search, and model selection.

 Data Preprocessing: Scikit-Learn provides tools for preprocessing and feature

engineering, including data scaling, normalization, encoding categorical variables,
imputing missing values, and feature selection. These preprocessing techniques are
essential for preparing data for machine learning algorithms.

 Pipeline: Scikit-Learn allows you to chain together multiple preprocessing steps and
machine learning algorithms into a single pipeline. This pipeline makes it easy to
encapsulate the entire machine learning workflow, from data preprocessing to model
training and prediction.

 Integration with NumPy and Pandas: Scikit-Learn seamlessly integrates with

NumPy arrays and Pandas DataFrame objects, allowing you to use these data
structures directly with Scikit-Learn algorithms.

Example

from sklearn.datasets import load_iris

from sklearn.model_selection import train_test_split

Department of CS&E,SJMIT,Chitradurga Page 28

 Automation & Analytics using Python
|Year: 2023-24

from sklearn.preprocessing import StandardScaler

from sklearn.svm import SVC

from sklearn.metrics import accuracy_score

# Load the Iris dataset

iris = load_iris()

X = iris.data

y = iris.target

# Split the dataset into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Standardize the features

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

X_test_scaled = scaler.transform(X_test)

# Train a Support Vector Machine classifier

clf = SVC(kernel='linear')

clf.fit(X_train_scaled, y_train)

# Make predictions on the testing set

y_pred = clf.predict(X_test_scaled)

# Evaluate the accuracy of the classifier

accuracy = accuracy_score(y_test, y_pred)

print('Accuracy:', accuracy)

Scikit-Learn is a powerful and versatile library for machine learning in Python. Its simple and
consistent interface, comprehensive set of algorithms, and extensive documentation make it
the go-to choice for many machine learning practitioners and researchers. Whether you're a
Department of CS&E,SJMIT,Chitradurga Page 29
 Automation & Analytics using Python
|Year: 2023-24

beginner exploring machine learning concepts or an experienced data scientist building

complex predictive models, Scikit-Learn provides the tools you need to tackle a wide range
of machine learning tasks with ease.

4.4.6 pylab

pylab is a module that combines the functionality of both matplotlib.pyplot (which is

typically imported as plt) and numpy (imported as np). It was historically used as a
convenient way to access the plotting functions of Matplotlib along with NumPy's array
operations in a single namespace.

However, it's generally considered a better practice to import matplotlib.pyplot and numpy
separately, as it provides better clarity and avoids potential namespace conflicts.

Using pylab:

import pylab as pl

# Generate some sample data

x = pl.linspace(0, 10, 100)

y = pl.sin(x)

# Plot the data

pl.plot(x, y)

Department of CS&E,SJMIT,Chitradurga Page 30

 Automation & Analytics using Python
|Year: 2023-24

pl.xlabel('X-axis')

pl.ylabel('Y-axis')

pl.title('Plot using Pylab')

pl.show()

In the above example, pylab is used to create a simple plot of a sine wave. It combines the
functionalities of both Matplotlib (plot, xlabel, ylabel, title, show) and NumPy (linspace, sin)
into a single namespace.

However, it's worth noting that using pylab is discouraged in favor of importing
matplotlib.pyplot and numpy separately. This helps in better organizing the code and
avoiding potential conflicts, especially in larger projects. Here's how the same example
would look using separate imports:

import numpy as np

import matplotlib.pyplot as plt

# Generate some sample data

x = np.linspace(0, 10, 100)

y = np.sin(x)

# Plot the data

plt.plot(x, y)

plt.xlabel('X-axis')

plt.ylabel('Y-axis')

plt.title('Plot using Matplotlib and NumPy')

plt.show()

This approach separates concerns more explicitly and is generally recommended for writing
clear and maintainable code.

4.4.7 SciPy
Department of CS&E,SJMIT,Chitradurga Page 31
 Automation & Analytics using Python
|Year: 2023-24

SciPy is an open-source Python library used for scientific and technical computing. It builds
on NumPy and provides a large number of higher-level functions for mathematical, scientific,
and engineering problems. SciPy includes modules for optimization, integration,
interpolation, eigenvalue problems, algebraic equations, differential equations, and many
other classes of problems. It is widely used in academia, research, and industry for various
computational tasks.

Key Features

 Optimization: SciPy provides functions for finding minima and maxima of functions,
including local and global optimization techniques. It includes solvers for linear
programming and root-finding algorithms.

 Integration: SciPy has tools for integrating functions, including single, double, and
multiple integrals, as well as ordinary differential equations (ODEs).

 Interpolation: SciPy provides functions for interpolation of data points in one and
two dimensions, including linear, spline, and polynomial interpolation.

 Linear Algebra: SciPy builds on NumPy’s linear algebra capabilities and includes
functions for solving linear systems, matrix factorizations, eigenvalue problems, and
other linear algebra tasks.

 Signal Processing: SciPy includes tools for signal processing, including filtering,
convolution, Fourier transforms, and spectral analysis.

 Statistics: SciPy provides functions for statistical distributions, statistical tests, and
descriptive statistics, making it useful for data analysis and hypothesis testing.

 Sparse Matrices: SciPy supports sparse matrix representations and operations, which
are essential for efficiently solving large-scale linear algebra problems.

Example

Optimization

Department of CS&E,SJMIT,Chitradurga Page 32

 Automation & Analytics using Python
|Year: 2023-24

import numpy as np

from scipy.optimize import minimize

# Define the objective function

def objective_function(x):

return x**2 + 2*x + 1

# Find the minimum of the function

result = minimize(objective_function, x0=0)

print('Minimum value:', result.fun)

print('Location of minimum:', result.x)

CONCLUSION

Department of CS&E,SJMIT,Chitradurga Page 33

 Automation & Analytics using Python
|Year: 2023-24

Python's robust libraries and frameworks, such as Pandas, NumPy, and SciPy, facilitate the
automation of repetitive and time-consuming tasks, enhancing productivity. Automated data
processing, cleaning, and manipulation streamline workflows, allowing professionals to focus
on more strategic activities. Python's capabilities in automation and analytics make it an
indispensable tool for modern data-driven environments. Its simplicity, combined with powerful
libraries and frameworks, facilitates the efficient handling of data, extraction of insights, and
deployment of scalable solutions. By leveraging Python, organizations can enhance their operational
efficiency, gain deeper insights from their data, and remain competitive in an increasingly data-centric
world.

Department of CS&E,SJMIT,Chitradurga Page 34