How to check for substantial change in two depth frames?

Question

My goal is to detect any change in frames(single channel. each pixel is depth value). On app init, I take average of all corresponding pixels of first 30 frames so one average background frame will be created. On new frames arrival I subtract each frame from saved background frame(mean frame of first 30 frames). Currently algorithm is: I take mean of first 30 frames say it bg_mean(scalar, not frame, say 2345). Then calculate mean of new frame and compare it with bg_mean with some threshold added to bg_mean (to avoid noise consideration) . But this method method does not give good results if distance is far. Are there any other methods ?

Answer 1

I suspect your threshold value is the issue. I'm not familiar with depth-cameras, but I'd assume the noise values might be impacted by the distance. To validate this, try changing/removing it to see if it improves the results.

Traditional image processing techniques will also update the history as time goes so large changes to the scene are not counted as changes in every subsequent frames. Your idea is essentially how it is done, but approaches vary in how they account for noise (in standard images you also have to account for shadows, which you can avoid with a depth camera)

I'd suggest looking into built-in algorithms with more sophisticated noise removal (https://docs.opencv.org/3.4/d/d38/tutorial_bgsegm_bg_subtraction.html). Although for images, I suspect this would work similarly for depths cameras.