Break a List into Chunks of Size N in Python

The goal here is to break a list into chunks of a specific size, such as splitting a list into sublists where each sublist contains n elements. For example, given a list [1, 2, 3, 4, 5, 6, 7, 8] and a chunk size of 3, we want to break it into the sublists [[1, 2, 3], [4, 5, 6], [7, 8]]. Let’s explore different approaches to accomplish this.

Using List Comprehension

List comprehension is an efficient method for chunking a list into smaller sublists. This method creates a list of lists, where each inner list represents a chunk of the original list of a given size.

a = [1, 2, 3, 4, 5, 6, 7, 8]
n = 3 

res = [a[i:i + n] for i in range(0, len(a), n)]
print(res) 

[[1, 2, 3], [4, 5, 6], [7, 8]]

Explanation:

• range(0, len(a), n) generates starting indices for each chunk, i.e., 0, 3, 6, etc.
• a[i:i + n] slices the list a starting at index i and ending at i + n (the chunk size).
• For each index in the range, the list is sliced into chunks of size n.

Using Slicing

This is a straightforward approach using a for loop to iterate over the list and slice it manually into chunks of size n. It’s simple to understand and works well for smaller datasets.

a = [1, 2, 3, 4, 5, 6, 7, 8]
n = 3  
res = [] 

for i in range(0, len(a), n):  # Slice list in steps of n
    res.append(a[i:i + n])

print(res)

[[1, 2, 3], [4, 5, 6], [7, 8]]

Explanation:

• For loop increments by n, ensuring each slice covers a chunk of size n.
• Each sliced chunk is appended to the chunks list.

Using itertools.islice

For very large lists, itertools.islice can be a memory-efficient way to create chunks without loading the entire list into memory. It allows you to process the list incrementally.

from itertools import islice

a = [1, 2, 3, 4, 5, 6, 7, 8]  
n = 3  

it = iter(a) 
res = [list(islice(it, n)) for _ in range((len(a) + n - 1) // n)]

print(res)

[[1, 2, 3], [4, 5, 6], [7, 8]]

Explanation:

• iter(a) converts list a into an iterator it for sequential access to elements.
• islice(it, n) fetches n elements at a time from the iterator it using islice.
• for _ in range((len(a) + n - 1) // n) iterates the required number of times to generate chunks, ensuring even an incomplete final chunk is included.

Using numpy.array_split

For handling larger or more complex datasets, numpy offers the array_split() function. It automatically handles unequal chunk sizes, making it particularly useful when the list size is not perfectly divisible by n.

import numpy as np
a = [1, 2, 3, 4, 5, 6, 7, 8]
n = 3

res = np.array_split(a, len(a) // n + (len(a) % n != 0))
res = [list(i) for i in res]
print(res)

[[np.int64(1), np.int64(2), np.int64(3)], [np.int64(4), np.int64(5), np.int64(6)], [np.int64(7), np.int64(8)]]

Explanation:

• array_split splits the list into chunks, automatically handling unequal chunk sizes.
• Each resulting numpy array is converted back into a Python list.