List comprehensions are one of the most remarkable features in Python. They come from Haskell, and not many other languages have them.

Programmers use them a lot because they're powerful, nice and short.

Every Python programmers should know about them and use them where appropriate.

What a list comprehension does, and how does it work?Permalink

List comprehension creates a new list from an existing iterable (e.g. list, set etc.). The existing iterable is an input for the list comprehension.

They may have conditions that determine if an item goes into the new list, and we can process (do something with) the item before it goes into the list—more on that in the next section.

The most minimal comprehension looks like this:

[thing for thing in some_iterable]

It has three parts:

1. Square brackets denote the start and the end of the list comprehension.
2. The expression whose result goes into the new list.
3. for loop that iterates through our iterable.

Ok, let's explain what is what:

Let's see it in action with some data:

scores = [6, 3, 8, 9, 34, 11, 18]

my_scores = [number for number in scores]

print(my_scores)
# outputs: [6, 3, 8, 9, 34, 11, 18]

In the code above, we create a simple list of numbers - scores. Then we use comprehension to create a new list out of scores.

It helps to read a comprehension starting from for and ending at the beginning:

For each number in scores, put the number into the new list.

We could achieve the same thing with this code:

new_scores = []
for number in scores:
    new_scores.append(number)

Here we can see how this comprehension translates to the for loop above:

As we can see, a list comprehension does the same thing as a for loop, but it needs only one line instead of three.

So a list comprehension is a for loop and list creation in one thing - with better, more concise syntax.

Comprehensions are also faster, but more on that later.

The above examples don't do much; they just create a copy of an existing list.

So when are comprehensions really useful? They shine when we need to do mapping or filtering (or both) of values in iterable and produce a new list.

MappingPermalink

Mapping (transforming) means that we want to take an item from the iterable, do something with it and return a new value.

Let's say we want to double all scores in our list.

scores = [6, 3, 8, 9, 34, 11, 18]

double_scores = [number * 2 for number in scores]

print(double_scores)
# outputs: [12, 6, 16, 18, 68, 22, 36]

Again what is before for goes to the new list:

And this is how it translates to the for loop:

We can also use an existing function in a comprehension:

def add100(score):
    return score+100

scores_100_more = [add100(number) for number in scores]

print(scores_100_more)
# outputs: [106, 103, 108, 109, 134, 111, 118]

The crucial thing to remember is this: the result of what is before the for in the comprehension goes into the new list. Thus, it must be something that returns a value - an expression.

If we call a function in the list comprehension, it should be a function that produces a meaningful value that we want to put into a new list. If we use a function that doesn't return anything, we will end with a new list full of None values.

def write_score(score):
    SomeStorage().write(score)

useless = [write_score(number) for number in scores]

print(useless)
# outputs: [None, None, None, None, None, None, None]

Sure our function could do something else, but unless it returns something that we want to use later, we should not use comprehension but rather a for loop.

List comprehension is for mapping and filtering.

FilteringPermalink

To use comprehension for filtering, we need to put if at the end like this:

scores = [6, 3, 8, 9, 34, 11, 18]

more_than_ten = [number for number in scores if number > 10]

With this, the part before the for will be executed and its result added to the new list only if the if part evaluates to True:

It is the equivalent of the following for loop:

more_than_ten = []
for number in scores:
    if number > 10:
        more_than_ten.append(number)

So now, while comprehension is still one short line, the for loop has grown to four lines.

This image shows how comprehension translates to the for loop with if:

Filter and map togetherPermalink

We can also filter and map at the same time.

Let's say we want to give one bonus point to all players with a score of more than 10:

scores = [6, 3, 8, 9, 34, 11, 18]

bonus = [number + 1 for number in scores if number > 10]
print(bonus)
# outputs: [35, 12, 19]

So this is what happened in the comprehension above:

1. It went through each number in scores.
2. When the if part on the right side was evaluated to True - if the number was more than 10 - it added 1 to it, and the result went into our new list.
3. When the if part on the right side was evaluated to False - for number 10 or less - it ignored that number and continued with the next one.

FormattingPermalink

When a lot is happening in our comprehension, we can always split it into multiple lines.

cards = [create_card(player)
         for player in all_players
         if is_top_player(player)]

Any form we like really:

cards = [create_card(player)
         for player
         in all_players
         if is_top_player(player)]

# or
cards = [create_card(player) for player in all_players
         if is_top_player(player)]

# or
cards = [create_card(player)
         for player in all_players if is_top_player(player)]

LimitationsPermalink

It is important to note that the new list is assigned to the variable only after comprehension is finished.

We don't have access to the list which is being created by the comprehension in the comprehension:

# This will not work `my_scores` does not exist yet
unique_names = [name  for name in all_names if name not in unique_names]

When to use comprehensionsPermalink

You should use comprehension only to create new list out of existing iterable - whether it is because we want to map/transform values or filter them.

In these cases it is preferred to use comprehensions.

When not to use comprehensionsPermalink

We should not use them for anything else except mapping and filtering.

They're not for general looping. For that, we should just use a for loop.

So we shouldn't do things like this:

[print(person.name) for person in people]

Yes, it will print names but:

1. It is slower - it creates a new list (for which the system needs to allocate memory), and then it adds None to it once for each person. So we'll end up with a list full of None values that is useless and gets discarted.
2. It is confusing. List comprehensions are for creating lists. If someone creates a list, it is because they want to use that list. It's not the case here. So a person looking at this will be confused about why have we done this.

SummaryPermalink

List comprehension:

• always creates a new list.
• can use any iterable as an input. So we can iterate through items in a tuple, set, string etc.
• should be used for filtering.
• should be used for mapping.
• is faster than writing a for loop.
• should not be used if we do not want to use or keep a resulting list.

Happy coding!

You might also like