Create a list from rows in Pandas dataframe

Python lists are one of the most versatile data structures, offering a range of built-in functions for efficient data manipulation. When working with Pandas, we often need to extract entire rows from a DataFrame and store them in a list for further processing. Unlike columns, which are easily accessible as Pandas Series, rows require different methods for extraction. Lets look at some different methods to do so:

Using DataFrame.values.tolist()

This method extracts all rows as a list of lists by converting the DataFrame into a NumPy array and then transforming it into a list. It is one of the fastest ways to convert a DataFrame into a list when performance is a key concern.

import pandas as pd

# create the dataframe
df = pd.DataFrame({'Date': ['10/2/2011', '11/2/2011', '12/2/2011', '13/2/2011'],
                   'Event': ['Music', 'Poetry', 'Theatre', 'Comedy'],
                   'Cost': [10000, 5000, 15000, 2000]})

# convert DataFrame to list
res = df.values.tolist()

print(res)

[['10/2/2011', 'Music', 10000], ['11/2/2011', 'Poetry', 5000], ['12/2/2011', 'Theatre', 15000], ['13/2/2011', 'Comedy', 2000]]

Explanation: This method extracts the underlying NumPy array of the DataFrame using df.values, which contains all rows as a structured array. The tolist() function then converts this array into a list of lists, where each row becomes a list.

Using DataFrame.to_numpy().tolist()

Similar to values.tolist(), this method explicitly converts the DataFrame into a NumPy array before converting it into a list. It provides better compatibility with newer Pandas versions.

import pandas as pd

# create the dataframe
df = pd.DataFrame({'Date': ['10/2/2011', '11/2/2011', '12/2/2011', '13/2/2011'],
                   'Event': ['Music', 'Poetry', 'Theatre', 'Comedy'],
                   'Cost': [10000, 5000, 15000, 2000]})

res = df.to_numpy().tolist()
print(res)

[['10/2/2011', 'Music', 10000], ['11/2/2011', 'Poetry', 5000], ['12/2/2011', 'Theatre', 15000], ['13/2/2011', 'Comedy', 2000]]

Explanation: Similar to values.tolist(), this method explicitly converts the DataFrame into a NumPy array using to_numpy().

Using DataFrame.to_dict(orient=’records’)

This method converts each row into a dictionary, where column names serve as keys. This is particularly useful when working with structured data that needs to retain column labels.

import pandas as pd

# Create the dataframe
df = pd.DataFrame({'Date': ['10/2/2011', '11/2/2011', '12/2/2011', '13/2/2011'],
                   'Event': ['Music', 'Poetry', 'Theatre', 'Comedy'],
                   'Cost': [10000, 5000, 15000, 2000]})

res = df.to_dict(orient='records')
print(res)

[{'Date': '10/2/2011', 'Event': 'Music', 'Cost': 10000}, {'Date': '11/2/2011', 'Event': 'Poetry', 'Cost': 5000}, {'Date': '12/2/2011', 'Event': 'Theatre', 'Cost': 15000}, {'Date': '13/2/2011', 'Event'...

Explanation: This method converts the DataFrame into a list of dictionaries, where each row is represented as a dictionary. The orient=’records’ argument ensures that each dictionary represents a row, with column names as keys and corresponding values as dictionary values.

Using DataFrame.itertuples()

This method iterates over each row as a named tuple, making it memory-efficient. It is useful for processing large datasets efficiently.

import pandas as pd

# Create the dataframe
df = pd.DataFrame({'Date': ['10/2/2011', '11/2/2011', '12/2/2011', '13/2/2011'],
                   'Event': ['Music', 'Poetry', 'Theatre', 'Comedy'],
                   'Cost': [10000, 5000, 15000, 2000]})

res = [list(row) for row in df.itertuples(index=False)]
print(res)

[['10/2/2011', 'Music', 10000], ['11/2/2011', 'Poetry', 5000], ['12/2/2011', 'Theatre', 15000], ['13/2/2011', 'Comedy', 2000]]

Explanation: itertuples() method iterates over each row as a named tuple, providing efficient row-wise access and using index=False ensures that the index column is not included in the tuples.