I’ve been tasked with developing software to (among other things) log temperature data from samples in a furnace. Generally the temperatures are very steady, but can, at times, experience dramatic changes.
I’ll be storing the data in SQLite, which is more than adequate for millions of rows, but this will be running on something like a Raspberry Pi with 8 GB of RAM. I’m doing testing now, but I’d rather not store data I don’t need to store.
The data should be easy to downsample regions of data without going below Nyquist, but when I search for “adaptive decimation” or “adaptive sampling,” I get some pretty weighty scholarly articles. I would have thought this wasn’t so hard to do, and widely available. I can do it after the fact, I don’t need to predict what my data is going to do. I don’t mind periodically going over older data to do the decimation.
It’s possible Pandas or some other Python library can do it with a simple call, so that might be a starting point for me (half the battle is figuring out what the technique is called). I’m writing this in Swift, so I don’t expect a ready-made solution, but if I knew what to implement, I could.
Any suggestions on how to accomplish this?