Android ImageReader motion detection

Question

I'm developing an app which has to be able to do motion detection based on androids camera2 API. So i'm working with an ImageReader and the corresponding OnImageAvailableListener. I have two target surfaces for the Frames of the CameraCaptureSession, the UI SurfaceTexture where I show the preview of the camera, and the ImageReaders surface.

I was kind of "forced" to use ImageFormat.YUV_420_88 for the same reason as mentioned in this question:

ImageReader makes Camera lag

I'm kind of new to this Format and i need to implement a motion detection.

My idea is to loop through Image.getPlanes()[i].getBuffer() pixel-wise and compare two images based on a certain threshold.

I have two Questions:

1. Is this a good idea? Is there maybe a better / more efficient way?

2. Which of the Plane(s) from the Format YUV_420_88 is best suited for such motion detection?

Answer 1

1. Is this a good idea? Is there maybe a better / more efficient way?

Yes, it is. There is no other way then compare continued frames to detect motion event.

2. Which of the Plane(s) from the Format YUV_420_88 is best suited for such motion detection?

I think it is Planes()[0].

My sample code:

mOnImageAvailableListener = new ImageReader.OnImageAvailableListener() {
        @Override
        public void onImageAvailable(ImageReader imageReader) {
            Image image = null;
            try {
                image = imageReader.acquireLatestImage();
                if (null == image) {
                    return;
                }
                ByteBuffer buffer = image.getPlanes()[0].getBuffer();
                byte[] bytes = new byte[buffer.capacity()];
                buffer.get(bytes);

                detectMotion(bytes);
            } finally {
                if (image != null) {
                    image.close();
                }
            }
        }

        private void detectMotion(byte[] frame) {
            if (null == frame) {
                return;
            }

            int ret = NativeUtil.detectMotion(frame);
            if (ret == 1) {
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        mIvMotion.setVisibility(View.VISIBLE);
                    }
                });
                Log.d(TAG, "Detected motion event");
            } else {
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        mIvMotion.setVisibility(View.INVISIBLE);
                    }
                });
            }
        }
    };