ive had this same issue before when dealing w/ sensor data that has occasional huge spikes and dips due to calibration issues! i found it really helpful to use a combination of
dropna() for removing obv bad points, followed by some rolling mean filtering. something like
['value'] = df[['value']].rolling(window=10).mean().bfill(axis='index')
can help smooth things out w/o losing too much data.
another gotcha i hit was forgetting to check the units of my time stamps - make sure theyre in a consistent format! it bit me once when timestamps were coming from two different sources and had slight discrepancies. always double-check those before jumping into interpolation or any other processing steps.
anyway, for your project!
> if you ever run into strong outliers like i did with sensor data,> try using z-score to identify them first!