Sliding Window for Data Streams

Question

I wish to add a sliding window to my data mining approach in order to handle a data stream. Does anyone have any helpful information with regards to implementing this? maybe a paper or some advice of your own?

I am working in c++ however any help would be appreciated!

Answer 1

It sounds like you want to estimate some slowly varying parameters of the stream. Your approach can be as straightforward as an actual moving window (keep the last n values so you'll know what to subtract when the trailing edge of the window moves forwards).

Or you could use a simple exponentially smoothed estimator:

estimate = k * estimate + (1-k) * measurement

or a more mathematically sophisticated filter embodying an a-priori theory of the dynamics of the stream, such as a http://en.wikipedia.org/wiki/Kalman_filter.