I have a list of dictionaries and each dictionary has a key of (let's say) 'type' which can have values of 'type1', 'type2', etc. My goal is to filter out these dictionaries into a list of the same dictionaries but only the ones of a certain "type". I think i'm just really struggling with list/dictionary comprehensions.
so an example list would look like:
exampleSet = [{'type':'type1'},{'type':'type2'},{'type':'type2'}, {'type':'type3'}]
i have a list of key values. lets say for example:
keyValList = ['type2','type3']
where the expected resulting list would look like:
expectedResult = [{'type':'type2'},{'type':'type2'},{'type':'type3'}]
I know i could do this with a set of for loops. I know there has to be a simpler way though. i found a lot of different flavors of this question but none that really fit the bill and answered the question. I would post an attempt at the answer... but they weren't that impressive. probably best to leave it open ended. any assistance would be greatly appreciated.
5 Answers
You can try a list comp
>>> exampleSet = [{'type':'type1'},{'type':'type2'},{'type':'type2'}, {'type':'type3'}]
>>> keyValList = ['type2','type3']
>>> expectedResult = [d for d in exampleSet if d['type'] in keyValList]
>>> expectedResult
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]
Another way is by using filter
>>> list(filter(lambda d: d['type'] in keyValList, exampleSet))
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]
Use filter, or if the number of dictionaries in exampleSet is too high, use ifilter of the itertools module. It would return an iterator, instead of filling up your system's memory with the entire list at once:
from itertools import ifilter
for elem in ifilter(lambda x: x['type'] in keyValList, exampleSet):
print elem
Trying a few answers from this post, I tested the performance of each answer.
As my initial guess, the list comprehension is way faster, the filter and list method is second and the pandas is third, by far.
defined variables:
import pandas as pd
exampleSet = [{'type': 'type' + str(number)} for number in range(0, 1_000_000)]
keyValList = ['type21', 'type950000']
1st - list comprehension
%%timeit
expectedResult = [d for d in exampleSet if d['type'] in keyValList]
60.7 ms ± 188 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
