New Union Operators to Merge Dictionaries in Python 3.9

merge (|) and update (|=) operators

two hands joined together to form the word “us”
Photo by Toa Heftiba on Unsplash

Merging Dictionaries in Python 3.9 Using the Union Operator

In Python 3.9, merge | and update |= operators have been added to the built-in dict class.

Let’s explore different ways to merge dictionaries before and after Python 3.9.


Different Ways to Merge Dictionaries Before Python 3.9

1.dict.update()

d1.update(d2) Update the dictionary d1 with the key/value pairs from d2, overwriting existing keys. Return None.

the sentence “last seen wins: last seen values will overwrite the overlapping keys” written in red letters
d1={'a':1,'b':2}
d2={'c':3,'b':9999}
d1.update(d2)
print (d1)
#Output:{'a': 1, 'b': 9999, 'c': 3}

Limitations

d1.update(d2) will modify the original dictionary d1. If the original dictionary need not be modified, create a copy of dictionary d1 and then update it.

d1={'a':1,'b':2}
d2={'c':3,'b':9999}
from copy import copy
d3=copy(d1)
d3.update(d2)
print (d3)
#Output:{'a': 1, 'b': 9999, 'c': 3}

2. Dictionary unpacking {**d1,**d2}

d3={**d1,**d2}

A double asterisk ** denotes dictionary unpacking.

It will expand the contents of dictionaries d1 and d2 as a collection of key-value pairs and update the dictionary d3. Keys that are common in d1 and d2 will contain values from d2.

the sentence “last seen wins: last seen values will overwrite the overlapping keys” written in red letters
d1={'a':1,'b':2}
d2={'c':3,'b':9999}
d3={**d1,**d2}
print (d3)
#Output:{'a': 1, 'b': 9999, 'c': 3}

Limitations

{**d1, **d2} ignores the types of mappings and always returns a dict.

3. collections.ChainMap

ChainMap: A ChainMap groups multiple dictionaries or other mappings together to create a single, updateable view.

collections.ChainMap(*maps)

Return type is collections.ChainMap. We can convert to dict using the dict() constructor.

the sentence “first seen wins: first seen values will overwrite the overlapping keys” written in red letters
d1={'a':1,'b':2}
d2={'c':3,'b':9999}
from collections import ChainMap
from collections import ChainMap
d3=ChainMap(d1,d2)
print (d3)
#Output:ChainMap({'a': 1, 'b': 2}, {'c': 3, 'b': 9999})
print (dict(d3))
#Output:{'c': 3, 'b': 2, 'a': 1}

Limitations

  1. It also ignores the types of mappings and always returns a dict.
  2. ChainMaps wrap their underlying dicts, so writing to the ChainMap will modify the original dict.

In the above-mentioned example, if we modify the ChainMap object, d1 will also be modified. d3=ChainMap(d1,d2)

d3['a']=555555
print (d1)
#Output:{'a': 555555, 'b': 2}

4. dict(d1,**d2)

d3=dict(d1,**d2)

d3 will contain key-value pairs from d1 and d2. Keys that are common in d1 and d2 will contain values from d2.

the sentence “last seen wins: last seen values will overwrite the overlapping keys” written in red letters
d1={'a':1,'b':2}
d2={'c':3,'b':9999}
from collections import ChainMap
d3=dict(d1,**d2)
print (d3)
#Output:{'a': 1, 'b': 9999, 'c': 3}

Limitations

d3=dict(d1,**d2) will work only when d2 is entirely string-keyed.

If int is given as a key in d2, it will raise TypeError.

d1={'a':1,'b':2}
d2={'a':99,1:3}
d3=dict(d1,**d2)
print (d3)
#Output:TypeError: keywords must be strings

New Ways Introduced in Python 3.9

Dict union

To quote specification 584,

“Dict union will return a new dict consisting of the left operand merged with the right operand, each of which must be a dict (or an instance of a dict subclass). If a key appears in both operands, the last-seen value (i.e. that from the right-hand operand) wins.”

the sentence “last seen wins: last seen values will overwrite the overlapping keys” written in red letters
d1={'a':1,'b':2}
d2={'c':3,'b':9999}
d3=d1|d2
print (d3)
#Output:{'a': 1, 'b': 9999, 'c': 3}

Augmented assignment version of dict union

d1|=d2 will modify d1 in place. It also accepts anything implementing the Mapping protocol (more specifically, anything with the keys and __getitem__ methods) or iterables of key-value pairs.

d1={'a':1,'b':2}
d2={'c':3,'b':9999}
d1|=d2
print (d1)
#Output:{'a': 1, 'b': 9999, 'c': 3}

Iterables of key-value pair are also accepted as the right operand.

d1={'a':1,'b':2}
d2=[('c',3),('b',9999)]
d1|=d2
print (d1)
#Output:{'a': 1, 'b': 9999, 'c': 3}

But in dict union, it accepts only dict or instances of dict.

d1={'a':1,'b':2}
d2=[('c',3),('b',9999)]
print (d1|d2)
#Output:TypeError: unsupported operand type(s) for |: 'dict' and 'list'

Limitations

1. Dict union is not commutative.

d1|d2 !=d2|d1

d1={'a':1,'b':2}
d2={'c':3,'b':9999}
d3=d1|d2
print (d3)
#Output: {'a': 1, 'b': 9999, 'c': 3}

d4={'a':1,'b':2}
d5={'c':3,'b':9999}
d6=d5|d4
print (d6)
#Output:{'c': 3, 'b': 2, 'a': 1}

print (d3==d6)
#Output:False

2. Repeated dict union is inefficient.

d1|d2|d3|d4|d5 creates and destroys three temporary mappings.

3. Dict union is lossy.

Dict union can lose data. Duplicate keys mean values will disappear. If a key appears in both operands, the last-seen value (i.e., that from the right-hand operand) wins. No other form of union is lossy.


Takeaways

table with three columns: expression, output, and last seen/first seen
Image by Author

d1.update(d2) and d1|=d2 both modify d1 in-place.

I hope that you have found this article helpful, and thanks for reading!


Resources

What’s new in Python 3.9

PEP-584 Add Union Operators to Dict

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s