November 03, 2003
Generator Expressions

PEP 289 -- Generator Expressions

I like this proposal. (Sorry, Hans!)

Python's List Comprehensions have long been a favourite of mine - powerful, and terse without being difficult to read or, uh, comprehend. (Those with only one for, that is. List comprehensions with multiple fors overflow my brain.)

But I must admit - I don't like the name much. 'Comprehension' - what does that have to do with anything?

List comprehensions always create the whole list. You don't always need the whole list at once, and if the list is big (or contains big objects), this might be a problem memory-wise. And if course, if your 'list' is infinite you can't materialise it at all! Python's generators are perfect for dealing with large or potentially infinite sequences.

PEP 289 proposes a list comprehension like syntax for creating generators - the generator expression.

Since the output of any finite generator can be turned into a list using the list() built-in, the existing list comprehension syntax will become merely syntax sugar for a generator expression so wrapped. But by wrapping in calls to dict() or tuple() or so on, one can create dictionaries and tuples as easily as lists.

Also, generator comprehensions are a natural fit with reduction functions like sum() - why materialise a whole list at once just to add up the numbers? New reduction functions are also planned to take advantage - though I hope these go into a module, rather than bulk out the built-ins too much.

All this, and a name that makes sense!

Posted to Python by Simon Brunning at November 03, 2003 05:11 PM
Post a comment

Email Address:



Remember info?