1

Data
Manipulation
1

Data
Manipulation
with pandas
2

Unless you do not

understand the data,
everything will be
confusable!

17
3

Introduction 18
3

Introduction 19
3

Introduction 20
4

Lecture I
INTRODUCTION
5

Introduction

22
5

Introduction

Data
Manipulation

23
5

Introduction

SELECT

Data
Manipulation

24
5

Introduction

SELECT

Data
Manipulation

INSERT

25
5

Introduction

SELECT

Data
Manipulation

INSERT
TRANS
FORM
26
5

Introduction

SELECT

Data
Manipulation

INSERT UPDATE
TRANS
FORM
27
5

Introduction
DELETE

SELECT

Data
Manipulation

INSERT UPDATE
TRANS
FORM
28
5

Introduction
DELETE

SELECT MERGE

Data
Manipulation

INSERT UPDATE
TRANS
FORM
29
5

Introduction DML command is used to modify

the table's existing data.

DELETE

SELECT MERGE

Data
Manipulation

INSERT UPDATE
TRANS
FORM
30
6

Introduction:

31
6

Introduction:

32
6

Introduction: Out of scope

33
7

• Getting data
REMINDER! • Describing data

34
10

Understanding data: Basics

the useful commands!

35
10

Understanding data: Basics

the useful commands!
df.head(X)

displays X
rows of the
dataframe

dataframe

36
10

Understanding data: Basics

the useful commands!
df.head(X) df.describe()

displays X Displays the

rows of the main statistics
dataframe about the
dataframe

dataframe

37
10

Understanding data: Basics

the useful commands!
df.head(X) df.describe()

displays X Displays the

rows of the main statistics
dataframe about the
dataframe

dataframe
dataframe

38
10

Understanding data: Basics

the useful commands!
df.head(X) df.describe() df.shape

displays X Displays the displays the

rows of the main statistics dimensionalit
dataframe about the y of the
dataframe dataframe

dataframe
dataframe

39
10

Understanding data: Basics

the useful commands!
df.head(X) df.describe() df.shape

displays X Displays the displays the

rows of the main statistics dimensionalit
dataframe about the y of the
dataframe dataframe

dataframe
dataframe a tuple

40
10

Understanding data: Basics

the useful commands!
df.head(X) df.describe() df.shape df.columns

displays X Displays the displays the The column

rows of the main statistics dimensionalit labels of the
dataframe about the y of the dataFrame
dataframe dataframe

dataframe
dataframe a tuple

41
10

Understanding data: Basics

the useful commands!
df.head(X) df.describe() df.shape df.columns

displays X Displays the displays the The column

rows of the main statistics dimensionalit labels of the
dataframe about the y of the dataFrame
dataframe dataframe

dataframe
an object
dataframe a tuple

42
11

• Definition!
• What we will learn!
DATA
MANIPULATION
43
12

Data Manipulation
Definition: Data manipulation is a tool to make the data more
understandable, readable, and organized.
12

Data Manipulation
Definition: Data manipulation is a tool to make the data more
understandable, readable, and organized.

‘’’There are no facts, there is no truth, just data to be manipulated.’’

Don Henley
13

Data Manipulation
Purposes:
1) Data Consistency – Consistency in data allows it to be read and understood. Data from various
sources may not have a unified view. However, data manipulation makes it easy for organizing and
storing it.
2) Data Projection – Data manipulation makes it possible to use historical data for projecting the
future and providing in-depth analysis, especially when it comes to financial transactions.
3) Create value from data – Data manipulation enables you to edit, delete, insert, or transform
data. Thus, you can do more with the available data. When you know how to use data, it becomes
better for your business.
4) Remove unnecessary data – Data can contain useless figures. Data manipulation allows you to
remove unusable data.

Source: https://www.digitalvidya.com/blog/data-manipulation/
14

Data Manipulation
Tips for Data Manipulation:
• The first step is forming a database from your data sources.
• Next, clean the data obtained from the source system before rearranging
and restructuring it.
• Import and construct a database that serves as the staging area.
• Combine or filter out information based on your business requirements.
• Lastly, leverage the manipulated data to garner valuable insights.
Source: https://www.astera.com/type/blog/data-manipulation-tools/
What will be learned!
• Exploratory Data Analysis (Lecture 2)
• Python codding (Lecture 3: EDA- Assignment 1 PubMed file)
• Indexing data and Selection (Lecture 4 on Python: Assignment 2)
• Time-Series & their manipulations (Lecture 5 on Python)
• Python codding (Lecture 6: Filtering, Indexing and Selecting a time series)
• Reshaping and Pivot Tables (Lecture 7 on Python)
• Rearranging, grouping data(split, apply, combine) transforming data(Lecture 8: Assignment 3)
• Concatenating data, Merging data, joining data and their comparison (Lecture 9)
• Handling categorical data, numerical data, and textual data (Lecture 10)

48
16

Learning Outcomes

1 2 3 4

Extract information -Explore different -Drop/Fill the null rows. Delete records from a
from a table techniques for merging -Modify the contents of database table, either
- Distinguish the a table individually or in
methods of left joins, groups
right joins, inner joins,
and outer joins

49