python - Quickest way to dedupe list in dict -

this question has answer here:

• how remove duplicates list in whilst preserving order? 33 answers

i have dict containing lists , need fast way dedupe lists.

i know how dedupe list in isolation using set() function, in case want fast way of iterating through dict, deduping each list on way.

hello = {'test1':[2,3,4,2,2,5,6], 'test2':[5,5,8,4,3,3,8,9]} 

i'd appear like;

hello = {'test1':[2,3,4,5,6], 'test2':[5,8,4,3,9]} 

though don't need have original order of lists preserved.

i've tried using set this, it's not quite correct (it's not iterating , i'm losing first key)

for key, value in hello.items(): goodbye = {key: set(value)} >>> goodbye {'test2': set([8, 9, 3, 4, 5])} 

edit: following pm 2ring's comment below, i'm populating dict differently avoid duplicates in first place. using lists, using sets prevents dupes appended default;

>>> my_numbers = {} >>> my_numbers['first'] = [1,2,2,2,6,5] >>> collections import defaultdict >>> final_list = defaultdict(set) >>> n in my_numbers['first']: final_list['test_first'].add(n) ...  >>> final_list['test_first'] set([1, 2, 5, 6]) 

as can see, final output deduped set, required.

it's not iterating wrong, you're assigning goodbye new dict each time. need assign empty dict assign values keys in each iteration.

goodbye = {} key, value in hello.items(): goodbye[key] = set(value) >>> goodbye {'test1': set([2, 3, 4, 5, 6]), 'test2': set([8, 9, 3, 4, 5])} 

also since sets don't preserve order, if want preserve order it's best make simple iterating function return new list skips on added values.

def uniquelist(li):     newlist = []     x in li:         if x not in newlist:             newlist.append(x)     return newlist   goodbye = {} key, value in hello.items(): goodbye[key] = uniquelist(value) >>> goodbye {'test1': [2, 3, 4, 5, 6], 'test2': [5, 8, 4, 3, 9]}