Using the itertools.batched function in Python 3.12

Python 3.12 introduced a new feature called itertools.batched() within the itertools module, which simplifies the process of creating iterators that return chunks (batches) of a given iterable. Before this function, developers typically had to write their own custom code to achieve the same result.

The Old Way

Prior to Python 3.12, there were a few common ways to create batched iterators. One of the most popular and Pythonic methods involved a custom generator function. This approach was highly flexible and efficient, as it processed data lazily (on demand).

Here’s a simple example of a generator function to batch an iterable:

Creating batched function before Python 3.12

def batched_old(iterable: list, n: int):
    iterator = iter(iterable)
    while True:
        chunk = tuple(itertools.islice(iterator, n))
        if not chunk:
            return
        yield chunk

This code uses itertools.islice() to create an iterator that returns a slice of the input iterable. It then converts this slice into a tuple to create the batch. The loop continues until islice() returns an empty sequence, at which point the generator stops.

Another common technique involved using a list comprehension or a similar pattern, but these methods were often less memory-efficient because they might load the entire iterable into memory before processing, which defeats the purpose of an iterator.

The New Way with itertools.batched()

With Python 3.12, you can now achieve the same result with a single line of code using itertools.batched(). The syntax is straightforward: itertools.batched(iterable, n), where iterable is the sequence you want to batch and n is the size of each batch.

Here’s how you use the new function:

Creating batches with itertools.batched in Python 3.12

import itertools

data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
batches = itertools.batched(data, 3)

for batch in batches:
    print(batch)

The code above will generate the following output:

Output of creating batches with itertools.batched in Python 3.12

(1, 2, 3)
(4, 5, 6)
(7, 8, 9)
(10,)

As you can see, the function automatically handles the remaining elements that don’t fit into a full batch, returning them as a final, smaller tuple.

The introduction of itertools.batched() standardizes a common programming pattern and makes code more readable and concise. It also improves code maintainability by reducing the need for custom, boilerplate functions that do the same thing. This is a small but welcome addition to the Python standard library that demonstrates the language’s ongoing evolution to support more modern and efficient coding practices.