How to Drop Rows with NaN Values in Pandas DataFrame?

In Pandas missing values are represented as NaN (Not a Number) which can lead to inaccurate analyses. One common approach to handling missing data is to drop rows containing NaN values using pandas. Below are some methods that can be used:

Method 1: Using dropna()

The dropna() method is the most straightforward way to remove rows with missing values. It scans through the DataFrame and drops any row that contains at least one NaN value. This method helps in maintaining data integrity as only complete records are used in analysis.

import pandas as pd
import numpy as np

data = {'Name': ['Alice', 'Bob', 'Charlie', 'David'],'Age': [25, np.nan, 35, 40],'City': ['New York', 'San Francisco', 'Los Angeles', np.nan],'Salary': [5000, 6000, 5500, 6200]}
df = pd.DataFrame(data)

a=df.dropna()
print(a)

Output:

using dropna

Here we removed all NaN value rows from dataset.

Method 2: Drop Rows Based on Specific Columns

The subset parameter allows you to specify columns that should be checked for NaN values allowing you to maintain rows that have missing values in less important columns.

import pandas as pd
import numpy as np

data = {'Name': ['Alice', 'Bob', 'Charlie', 'David', np.nan, np.nan],'Age': [25, np.nan, 35, 40, np.nan, np.nan],'City': ['New York', 'San Francisco', 'Los Angeles', np.nan, np.nan, np.nan],'Salary': [5000, 6000, 5500, 6200, np.nan, np.nan]}
df = pd.DataFrame(data)

df_clean = df.dropna(subset=['Name', 'Age'])
print(df_clean)

Output:

Drop rows based on specific coloumn

Here we removed rows with NaN values present in Name and Age column.

Method 3: In-Place Modification with inplace=True

By default dropna() returns a new DataFrame and leaves the original dataframe unchanged. Use inplace=True to modify the original DataFrame directly. This approach save memory and processing time when working with large datasets.

import pandas as pd
import numpy as np

data = {'Name': ['Alice', 'Bob', 'Charlie', 'David', np.nan, np.nan],'Age': [25, np.nan, 35, 40, np.nan, np.nan],'City': ['New York', 'San Francisco', 'Los Angeles', np.nan, np.nan, np.nan],'Salary': [5000, 6000, 5500, 6200, np.nan, np.nan]}
df = pd.DataFrame(data)

df.dropna(inplace=True)
print(df)

Output:

In place modification

The dropna() function offers various parameters that provide flexibility in handling missing data.

shardul_singh_tomar

Follow

Improve

Article Tags :

• AI-ML-DS
• Pandas
• Python
• Python pandas-dataFrame
• Python-pandas

+1 More

Practice Tags :

• python