The common way is to get a unique collection of items is to use a set [Refer to this][1]
>>> t = [1, 2, 3, 1, 2, 5, 6, 7, 8] >>> t [1, 2, 3, 1, 2, 5, 6, 7, 8] >>> list(set(t)) [1, 2, 3, 5, 6, 7, 8] >>> s = [1, 2, 3] >>> list(set(t) - set(s)) [8, 5, 6, 7]

[1]: https://stackoverflow.com/questions/7961363/removing-duplicates-in-lists

The common approach to get a unique collection of items is to use a [`set`](http://docs.python.org/3/library/stdtypes.html#set-types-set-frozenset). Sets are *unordered* collections of *distinct* objects. To create a set from any iterable, you can simply pass it to the built-in [`set()`](http://docs.python.org/3/library/functions.html#func-set) function. If you later need a real list again, you can similarly pass the set to the [`list()`](http://docs.python.org/3/library/functions.html#func-list) function.
The following example should cover whatever you are trying to do:
>>> t = [1, 2, 3, 1, 2, 5, 6, 7, 8] >>> t [1, 2, 3, 1, 2, 5, 6, 7, 8] >>> list(set(t)) [1, 2, 3, 5, 6, 7, 8] >>> s = [1, 2, 3] >>> list(set(t) - set(s)) [8, 5, 6, 7]
As you can see from the example result, **the original order is not maintained**. As mentioned above, sets themselves are unordered collections, so the order is lost. When converting a set back to a list, an arbitrary order is created.
If order is important to you, then you will have to use a different mechanism. A very common solution for this is to rely on [`OrderedDict`](https://docs.python.org/3/library/collections.html#collections.OrderedDict) to keep the order of keys during insertion:
>>> from collections import OrderedDict >>> list(OrderedDict.fromkeys(t)) [1, 2, 3, 5, 6, 7, 8]
Note that this has the overhead of creating a dictionary first, and then creating a list from it. So if you don’t actually need to preserve the order, you’re better off using a set. Check out [this question](https://stackoverflow.com/q/480214/216074) for more details and alternative ways to preserve the order when removing duplicates.
---
Finally note that both the `set` as well as the `OrderedDict` solution require your items to be *hashable*. This usually means that they have to be immutable. If you have to deal with items that are not hashable (e.g. list objects), then you will have to use a slow approach in which you will basically have to compare every item with every other item in a nested loop.