I have a large data set (10's of billions) of data points (doubles) that I need to display on a chart. Since displaying all of the data at once is not useful, I was looking for an algorithm that will help me pick the best N points from the whole set.
I am currently doing Systematic Sampling to reduce the dataset. Any suggestions on how to improve on it? Thanks.
Update: The data is 16 bit signed numbers signifying the amplitude of a waveform. So they can range in value from -32,768 to 32,767. I want to capture the peaks and valley so that the N points picked to display from the whole set give an approximation of the entire set.