Python | Remove duplicate lists in tuples (Preserving Order)
Last Updated :
21 Apr, 2023
Sometimes, while working with records, we can have a problem in which we need to remove duplicate records. This kind of problem is common in web development domain. Let’s discuss certain ways in which this task can be performed.
Method #1 : Using list comprehension + set()
In this method, we test for each list as it appears and add it to set so that it’s repeated occurrence can be avoided and then this is added to newly maintained unique tuple, removing duplicates.
Python3
test_tup = ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
print("The original tuple is : " + str(test_tup))
temp = set()
res = [ele for ele in test_tup if not(tuple(ele) in temp or temp.add(tuple(ele)))]
print("The unique lists tuple is : " + str(res))
|
Output :
The original tuple is : ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
The unique lists tuple is : [[4, 7, 8], [1, 2, 3], [9, 10, 11]]
Time complexity: O(n)
Auxiliary Space: O(n)
Method #2 : Using OrderedDict() + tuple()
The combination of above functions can also be used to perform this particular task. In this, we convert the tuple to OrderedDict(), which automatically removes the duplicate elements and then construct a new tuple list using tuple().
Python3
from collections import OrderedDict
test_tup = ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
print("The original tuple is : " + str(test_tup))
res = list(OrderedDict((tuple(x), x) for x in test_tup).values())
print("The unique lists tuple is : " + str(res))
|
Output :
The original tuple is : ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
The unique lists tuple is : [[4, 7, 8], [1, 2, 3], [9, 10, 11]]
Method #3: Using recursive method.
Python3
def remove_duplicates(tup, result, seen):
if not tup:
return result
if tuple(tup[0]) not in seen:
result.append(tup[0])
seen.add(tuple(tup[0]))
return remove_duplicates(tup[1:], result, seen)
test_tup = ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
result = []
seen = set()
res = remove_duplicates(test_tup, result, seen)
print("The original tuple is : " + str(test_tup))
print("The unique lists tuple is : " + str(res))
|
Output
The original tuple is : ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
The unique lists tuple is : [[4, 7, 8], [1, 2, 3], [9, 10, 11]]
Time Complexity: O(n)
Auxiliary Space: O(n)
Method #4:Using a loop and a set:
Algorithm:
1.Create an empty list called “result” and an empty set called “seen”.
2.For each sublist in the input tuple:
a. Convert the sublist to a tuple called “t”.
b. If “t” is not in the “seen” set, append the original sublist to the “result” list and add “t” to the “seen” set.
3.Convert the “result” list to a tuple and return it.
Python3
def remove_duplicates(tup):
result = []
seen = set()
for sublist in tup:
t = tuple(sublist)
if t not in seen:
result.append(sublist)
seen.add(t)
return tuple(result)
test_tup = ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
unique_tup = remove_duplicates(test_tup)
print("Original tuple:", test_tup)
print("Tuple with duplicate lists removed:", unique_tup)
|
Output
Original tuple: ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
Tuple with duplicate lists removed: ([4, 7, 8], [1, 2, 3], [9, 10, 11])
Time complexity: O(n*m) where n is the number of sublists in the input tuple and m is the length of the largest sublist. This is because we need to iterate over each sublist and create a tuple from it, and checking membership in a set takes constant time on average.
Auxiliary Space: O(n*m) because we need to store the input tuple, the output list, and the set of seen tuples. In the worst case where all sublists are unique, the size of the output list will be equal to the size of the input tuple, so the space complexity will be proportional to the size of the input tuple.
Method #5: Using loop and if-else statement
Algorithm:
- Initialize an empty list res to store unique lists.
- Loop through each list i in the given tuple test_tup.
- If the list i is not already in the res list, then append i to the res list.
- Return the res list as the result.
Python3
test_tup = ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
res = []
for i in test_tup:
if i not in res:
res.append(i)
print("The unique lists tuple is : " + str(res))
|
Output
The unique lists tuple is : [[4, 7, 8], [1, 2, 3], [9, 10, 11]]
Time Complexity:
The loop runs n times, where n is the length of the input tuple.
The list append operation and the list membership test both have O(1) time complexity on average.
Therefore, the overall time complexity of the algorithm is O(n) on average.
Auxiliary Space:
The space used by the res list is O(k), where k is the number of unique lists in the input tuple.
The space used by other variables is constant.
Therefore, the overall space complexity of the algorithm is O(k).
Method 6: Using generator function
Define a generator function that takes a tuple as input.
Initialize an empty set to store the seen tuples.
Loop through the original tuple.
Convert the current list to a tuple and check if it is already in the set.
If it is not in the set, yield the current list and add the tuple to the set.
Return the generator function.
Python3
def remove_duplicates(tup):
seen = set()
for lst in tup:
tup_lst = tuple(lst)
if tup_lst not in seen:
yield lst
seen.add(tup_lst)
test_tup = ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
res = tuple(remove_duplicates(test_tup))
print("The original tuple is : " + str(test_tup))
print("The unique lists tuple is : " + str(res))
|
Output
The original tuple is : ([4, 7, 8], [1, 2, 3], [4, 7, 8], [9, 10, 11], [1, 2, 3])
The unique lists tuple is : ([4, 7, 8], [1, 2, 3], [9, 10, 11])
This method has a time complexity of O(n), where n is the length of the input tuple, since we only loop through the tuple once.
This method has an auxiliary space complexity of O(n), where n is the length of the input tuple, since we need to store the unique tuples in a set.
Similar Reads
Python - Remove Kth Index Duplicates in Tuple
Sometimes, while working with Python records, we can have a problem in which we need to remove all the tuples, which have similar Kth index elements in list of records. This kind of problem is common in day-day and web development domain. Let's discuss certain ways in which this task can be performe
7 min read
Python Program to Remove duplicate tuples irrespective of order
Given a list of binary tuples, the task is to write a Python program to remove all tuples that are duplicates irrespective of order, i.e delete if contains similar elements, irrespective of order. Input : test_list = [(4, 6), (1, 2), (9, 2), (2, 1), (5, 7), (6, 4), (9, 2)]Output : [(1, 2), (5, 7), (
6 min read
Python List: Remove Duplicates And Keep The Order
While lists provide a convenient way to manage collections of data, duplicates within a list can sometimes pose challenges. In this article, we will explore different methods to remove duplicates from a Python list while preserving the original order. Using dict.fromkeys()dict.fromkeys() method crea
2 min read
Python | Remove duplicate tuples from list of tuples
Given a list of tuples, Write a Python program to remove all the duplicated tuples from the given list. Examples: Input : [(1, 2), (5, 7), (3, 6), (1, 2)] Output : [(1, 2), (5, 7), (3, 6)] Input : [('a', 'z'), ('a', 'x'), ('z', 'x'), ('a', 'x'), ('z', 'x')] Output : [('a', 'z'), ('a', 'x'), ('z', 'x
5 min read
Python | Remove duplicates based on Kth element tuple list
Sometimes, while working with records, we may have a problem in which we need to remove duplicates based on Kth element of a tuple in the list. This problem has application in domains that uses records as input. Let's discuss certain ways in which this problem can be solved. Method #1: Using loop Th
8 min read
Removing Duplicates of a List of Sets in Python
Dealing with sets in Python allows for efficient handling of unique elements, but when working with a list of sets, you may encounter scenarios where duplicates need to be removed. In this article, we will see how we can remove duplicates of a list of sets in Python. Remove Duplicates of a List of S
2 min read
Python | Remove duplicates from nested list
The task of removing duplicates many times in the recent past, but sometimes when we deal with the complex data structure, in those cases we need different techniques to handle this type of problem. Let's discuss certain ways in which this task can be achieved. Method #1 : Using sorted() + set() Thi
5 min read
Python program to remove duplicate elements index from other list
Given two lists, the task is to write a Python program to remove all the index elements from 2nd list which are duplicate element indices from 1st list. Examples: Input : test_list1 = [3, 5, 6, 5, 3, 7, 8, 6], test_list2 = [1, 7, 6, 3, 7, 9, 10, 11] Output : [1, 7, 6, 9, 10] Explanation : 3, 7 and 1
7 min read
Print a List of Tuples in Python
The task of printing a list of tuples in Python involves displaying the elements of a list where each item is a tuple. A tuple is an ordered collection of elements enclosed in parentheses ( ), while a list is an ordered collection enclosed in square brackets [ ]. Using print()print() function is the
2 min read
Python | Removing duplicates from tuple
Many times, while working with Python tuples, we can have a problem removing duplicates. This is a very common problem and can occur in any form of programming setup, be it regular programming or web development. Let's discuss certain ways in which this task can be performed. Method #1 : Using set()
4 min read