QBoard » Supporting Tech Stack » IOT » How do I merge two dictionaries in a single expression (taking union of dictionaries)?

How do I merge two dictionaries in a single expression (taking union of dictionaries)?

  • I have two Python dictionaries, and I want to write a single expression that returns these two dictionaries, merged (i.e. taking the union). The update() method would be what I need, if it returned its result instead of modifying a dictionary in-place.
    >>> x = {'a': 1, 'b': 2}
    >>> y = {'b': 10, 'c': 11}
    >>> z = x.update(y)
    >>> print(z)
    None
    >>> x
    {'a': 1, 'b': 10, 'c': 11}
    

    How can I get that final merged dictionary in z, not x?

    (To be extra-clear, the last-one-wins conflict-handling of dict.update() is what I'm looking for as well.)

      September 7, 2021 5:02 PM IST
    0
  • In your case, what you can do is:

    z = dict(list(x.items()) + list(y.items()))
    


    This will, as you want it, put the final dict in z, and make the value for key b be properly overridden by the second (y) dict's value:


    >>> x = {'a':1, 'b': 2}
    >>> y = {'b':10, 'c': 11}
    >>> z = dict(list(x.items()) + list(y.items()))
    >>> z
    {'a': 1, 'c': 11, 'b': 10}


    If you use Python 2, you can even remove the list() calls. To create z:

    >>> z = dict(x.items() + y.items())
    >>> z
    {'a': 1, 'c': 11, 'b': 10}

     

    If you use Python version 3.9.0a4 or greater, then you can directly use:

    x = {'a':1, 'b': 2}
    y = {'b':10, 'c': 11}
    z = x | y
    print(z)

     

    {'a': 1, 'c': 11, 'b': 10}
    



      September 13, 2021 1:20 PM IST
    0
  • I know this does not really fit the specifics of the questions ("one liner"), but since none of the answers above went into this direction while lots and lots of answers addressed the performance issue, I felt I should contribute my thoughts.

    Depending on the use case it might not be necessary to create a "real" merged dictionary of the given input dictionaries. A view which does this might be sufficient in many cases, i. e. an object which acts like the merged dictionary would without computing it completely. A lazy version of the merged dictionary, so to speak.

    In Python, this is rather simple and can be done with the code shown at the end of my post. This given, the answer to the original question would be:

    z = MergeDict(x, y)
    

     

    When using this new object, it will behave like a merged dictionary but it will have constant creation time and constant memory footprint while leaving the original dictionaries untouched. Creating it is way cheaper than in the other solutions proposed.

    Of course, if you use the result a lot, then you will at some point reach the limit where creating a real merged dictionary would have been the faster solution. As I said, it depends on your use case.

    If you ever felt you would prefer to have a real merged dict, then calling dict(z) would produce it (but way more costly than the other solutions of course, so this is just worth mentioning).

    You can also use this class to make a kind of copy-on-write dictionary:

    a = { 'x': 3, 'y': 4 }
    b = MergeDict(a)  # we merge just one dict
    b['x'] = 5
    print b  # will print {'x': 5, 'y': 4}
    print a  # will print {'y': 4, 'x': 3}

     

    Here's the straight-forward code of Merge Dict:

    class MergeDict(object):
      def __init__(self, *originals):
        self.originals = ({},) + originals[::-1]  # reversed
    
      def __getitem__(self, key):
        for original in self.originals:
          try:
            return original[key]
          except KeyError:
            pass
        raise KeyError(key)
    
      def __setitem__(self, key, value):
        self.originals[0][key] = value
    
      def __iter__(self):
        return iter(self.keys())
    
      def __repr__(self):
        return '%s(%s)' % (
          self.__class__.__name__,
          ', '.join(repr(original)
              for original in reversed(self.originals)))
    
      def __str__(self):
        return '{%s}' % ', '.join(
            '%r: %r' % i for i in self.iteritems())
    
      def iteritems(self):
        found = set()
        for original in self.originals:
          for k, v in original.iteritems():
            if k not in found:
              yield k, v
              found.add(k)
    
      def items(self):
        return list(self.iteritems())
    
      def keys(self):
        return list(k for k, _ in self.iteritems())
    
      def values(self):
        return list(v for _, v in self.iteritems())



      September 14, 2021 1:26 PM IST
    0
  • I wanted something similar, but with the ability to specify how the values on duplicate keys were merged, so I hacked this out (but did not heavily test it). Obviously this is not a single expression, but it is a single function call.
    def merge(d1, d2, merge_fn=lambda x,y:y):
        """
        Merges two dictionaries, non-destructively, combining 
        values on duplicate keys as defined by the optional merge
        function.  The default behavior replaces the values in d1
        with corresponding values in d2.  (There is no other generally
        applicable merge strategy, but often you'll have homogeneous 
        types in your dicts, so specifying a merge technique can be 
        valuable.)
    
        Examples:
    
        >>> d1
        {'a': 1, 'c': 3, 'b': 2}
        >>> merge(d1, d1)
        {'a': 1, 'c': 3, 'b': 2}
        >>> merge(d1, d1, lambda x,y: x+y)
        {'a': 2, 'c': 6, 'b': 4}
    
        """
        result = dict(d1)
        for k,v in d2.iteritems():
            if k in result:
                result[k] = merge_fn(result[k], v)
            else:
                result[k] = v
        return result
      September 17, 2021 10:50 PM IST
    0