Python Program to Remove Duplicates from List

Write a Python Program to Remove all the duplicates from the given list. The Python set won’t allow duplicates, so we can convert the list to set, and then convert it back to the list will remove the list duplicates.

# Remove Duplicates from List

dupList = [1, 2, 3, 2, 4, 8, 9, 1, 7, 6, 4, 5]
print("List Items = ", dupList)

uniqSet = set(dupList)
uniqList = list(uniqSet)
   
print("List Items after removing Duplicates = ", uniqList)

Python Remove Duplicates in a List using set output

List Items =  [1, 2, 3, 2, 4, 8, 9, 1, 7, 6, 4, 5]
List Items after removing Duplicates =  [1, 2, 3, 4, 5, 6, 7, 8, 9]

Python Program to Remove Duplicate Items from List

This Python program allows entering the list size and items. The for loop will iterate the dupList items. The if statement with not in operator checks whether the value is not present in the uniqList. If True, append that value to uniqList.

# Remove Duplicates from List

dupList = []

listNumber = int(input("Enter the Total List Items = "))
for i in range(1, listNumber + 1):
    listValue = int(input("Enter the %d List Item = " %i))
    dupList.append(listValue)

print("List Items = ", dupList)

uniqList = []

for val in dupList:
    if val not in uniqList:
        uniqList.append(val)
   
print("List Items after removing Duplicates = ", uniqList)
Python Program to Remove Duplicates from List 2

In this example, we used the Python list comprehension to remove duplicate items from the list. This code is the same as the above example, but we used the list comprehension concept.

# Remove Duplicates from List

dupList = [1, 2, 5, 8, 1, 9, 11, 5, 22, 6, 2, 8, 14]

print("List Items = ", dupList)

uniqList = []
[uniqList.append(i) for i in dupList if i not in uniqList]

print("List Items after removing Duplicates = ", uniqList)
List Items =  [1, 2, 5, 8, 1, 9, 11, 5, 22, 6, 2, 8, 14]
List Items after removing Duplicates =  [1, 2, 5, 8, 9, 11, 22, 6, 14]

In this example, we imported the OrderedDict from collections and using the fromkeys function to remove duplicates. Don’t forget to convert the result to a list.

# Remove Duplicates from List

from collections import OrderedDict

dupList = [8, 1, 9, 2, 8, 4, 9, 11, 5, 22, 6, 4, 8]

print("List Items = ", dupList)

uniqList = OrderedDict.fromkeys(dupList)

print("List Items after removing Duplicates = ", list(uniqList))

Remove Duplicates in a List using OrderedDict from collections output

List Items =  [8, 1, 9, 2, 8, 4, 9, 11, 5, 22, 6, 4, 8]
List Items after removing Duplicates =  [8, 1, 9, 2, 4, 11, 5, 22, 6]

Both the numpy and pandas module has the unique function to remove duplicates, so we used the same and converted the result to a list. To convert the result, we used the tolist() function.

# Remove Duplicates from List

import numpy as np
import pandas as pd

dupList = [1, 2, 2, 4, 1, 5, 6, 8, 6, 8, 9, 7, 4]
print("List Items = ", dupList)

uniqList = np.unique(dupList).tolist()
print("List Items after removing Duplicates = ", uniqList)

uniqList2 = pd.unique(dupList).tolist()
print("List Items after removing Duplicates = ", uniqList2)

Remove Duplicates in a List using numpy unique function output

List Items =  [1, 2, 2, 4, 1, 5, 6, 8, 6, 8, 9, 7, 4]
List Items after removing Duplicates =  [1, 2, 4, 5, 6, 7, 8, 9]
List Items after removing Duplicates =  [1, 2, 4, 5, 6, 8, 9, 7]

Python Program to Remove Duplicates from List using enumerate.

# Remove Duplicates from List

from collections import OrderedDict

dupList = [1, 2, 3, 2, 4, 1, 5, 6, 5, 8, 7, 9, 8]

print("List Items = ", dupList)

uniqList = [val for x, val in enumerate(dupList) if val not in dupList[:x]]

print("List Items after removing Duplicates = ", uniqList)

Remove Duplicates in a List using enumerate output

List Items =  [1, 2, 3, 2, 4, 1, 5, 6, 5, 8, 7, 9, 8]
List Items after removing Duplicates =  [1, 2, 3, 4, 5, 6, 8, 7, 9]