In a time series (ordered tuples), what's the most efficient way to find the first time a criterion is met?
In particular, what's the most efficient way to determine when a value goes over 100 for the value of a column in a pandas data frame?
I was hoping for a clever vectorized solution, and not having to use df.iterrows().
For example, for price or count data, when a value exceeds 100. I.e. df['col'] > 100.
price
date
2005-01-01 98
2005-01-02 99
2005-01-03 100
2005-01-04 99
2005-01-05 98
2005-01-06 100
2005-01-07 100
2005-01-08 98
but for potentially very large series. Is it better to iterate (slow) or is there a vectorized solution?
A df.iterrows() solution could be:
for row, ind in df.iterrows():
if row['col'] > value_to_check:
breakpoint = row['value_to_record'].loc[ind]
return breakpoint
return None
But my question is more about efficiency (potentially, a vectorized solution that will scale well).
2 Answers
Try this: "> 99"
df[df['price'].gt(99)].index[0]
returns "2", the second index row.
all row indexes greater than 99
df[df['price'].gt(99)].index
Int64Index([2, 5, 6], dtype='int64')
This will return the index value of the first occurrence of 100 in the series:
index_value = (df['col'] - 100).apply(abs).idxmin()
If there is no value exactly 100, it should return the index of the closest value.
