Writing Memory Efficient Programs Using Generators in Python

Merge Two Lists Without Duplicates - Python

Last Updated : 08 Feb, 2025

We are given two lists containing elements and our task is to merge them into a single list while ensuring there are no duplicate values. For example: a = [1, 2, 3, 4] and b = [3, 4, 5, 6] then the merged list should be: [1, 2, 3, 4, 5, 6]

Using set()

set() function automatically removes duplicates making it the most efficient way to merge two lists while keeping only unique elements.

Python

a = [1, 2, 3, 4]  
b = [3, 4, 5, 6]  

res = list(set(a + b))  

print(res)

Output

[1, 2, 3, 4, 5, 6]

Explanation:

a + b merges both lists and set(a + b) removes duplicates.
list(set(...)) converts the set back into a list.

Using Dictionary Keys (dict.fromkeys())

Since dictionary keys are unique, we can use dict.fromkeys() to remove duplicates while preserving the order of elements.

Python

a = [1, 2, 3, 4]  
b = [3, 4, 5, 6]  

res = list(dict.fromkeys(a + b))  

print(res)

Output

[1, 2, 3, 4, 5, 6]

Explanation:

a + b merges both lists and dict.fromkeys(a + b) creates a dictionary where keys are the merged elements (removing duplicates).
list(...) converts the dictionary keys back into a list preserving order.

Using a for Loop

We can manually iterate through both lists and add elements to a new list only if they are not already present.

Python

a = [1, 2, 3, 4]  
b = [3, 4, 5, 6]  

res = []  
for x in a + b:  
    if x not in res:  
        res.append(x)  

print(res)

Output

[1, 2, 3, 4, 5, 6]

Explanation:

We initialize an empty list result and iterate through a + b, check if x is already in result.
If x is not in result then only we append it.

Using List Comprehension with set

This method combines a set for fast lookups with a list comprehension to preserve order efficiently.

Python

a = [1, 2, 3, 4]  
b = [3, 4, 5, 6]  

seen = set()  
res = [x for x in a + b if x not in seen and not seen.add(x)]  

print(res)

Output

[1, 2, 3, 4, 5, 6]

Explanation:

We initialize an empty set() called seen to track unique elements.
We iterate through a + b using list comprehension and if x is not in seen then it is added to result, seen.add(x) adds x to seen ensuring duplicates are ignored.

Writing Memory Efficient Programs Using Generators in Python

H

harshasi1eet

Improve

Article Tags :

Practice Tags :

Similar Reads

Python - Merging duplicates to list of list

We are given a list we need to merge the duplicate to lists of list. For example a list a=[1,2,2,3,4,4,4,5] we need to merge all the duplicates in lists so that the output should be [[2,2],[4,4,4]]. This can be done by using multiple functions like defaultdict from collections and Counter and variou

Python - Merge Dictionaries List with duplicate Keys

Given two List of dictionaries with possible duplicate keys, write a Python program to perform merge. Examples: Input : test_list1 = [{"gfg" : 1, "best" : 4}, {"geeks" : 10, "good" : 15}, {"love" : "gfg"}], test_list2 = [{"gfg" : 6}, {"better" : 3, "for" : 10, "geeks" : 1}, {"gfg" : 10}] Output : [{

Python | Merge two lists alternatively

Given two lists, write a Python program to merge the given lists in an alternative fashion, provided that the two lists are of equal length. Examples: Input : lst1 = [1, 2, 3] lst2 = ['a', 'b', 'c'] Output : [1, 'a', 2, 'b', 3, 'c'] Input : lst1 = ['name', 'alice', 'bob'] lst2 = ['marks', 87, 56] Ou

Python List: Remove Duplicates And Keep The Order

While lists provide a convenient way to manage collections of data, duplicates within a list can sometimes pose challenges. In this article, we will explore different methods to remove duplicates from a Python list while preserving the original order. Using dict.fromkeys()dict.fromkeys() method crea

Python Merge Dictionaries with Same Keys

When working with dictionaries in Python, you should understand that they are mutable, unordered collections of key-value pairs. This flexibility allows us to quickly merge dictionaries, but what happens when two dictionaries have the same key? Python allows us to manage this situation. In this arti

Python | Remove duplicates from nested list

The task of removing duplicates many times in the recent past, but sometimes when we deal with the complex data structure, in those cases we need different techniques to handle this type of problem. Let's discuss certain ways in which this task can be achieved. Method #1 : Using sorted() + set() Thi

Python | Merge Python key values to list

Sometimes, while working with Python, we might have a problem in which we need to get the values of dictionary from several dictionaries to be encapsulated into one dictionary. This type of problem can be common in domains in which we work with relational data like in web developments. Let's discuss

Python | Merge two lists into list of tuples

The task of merging two lists into a list of tuples involves combining corresponding elements from both lists into paired tuples. For example, given two lists like a = [1, 2, 3] and b = ['a', 'b', 'c'], the goal is to merge them into a list of tuples, resulting in [(1, 'a'), (2, 'b'), (3, 'c')]. Usi

Merge Dictionaries without Overwriting in Python

Merging dictionaries in Python is a common operation. In this article, we will see how we can append two dictionaries so that they don't overwrite each other in Python. Append two dictionaries so they don't Overwrite Each OtherBelow are some of the ways by which we can append two dictionaries in Pyt

Python | Remove duplicate lists in tuples (Preserving Order)

Sometimes, while working with records, we can have a problem in which we need to remove duplicate records. This kind of problem is common in web development domain. Let's discuss certain ways in which this task can be performed. Method #1 : Using list comprehension + set() In this method, we test fo