Video recording and onPreviewFrame callback at the same time

Question

I'm trying to record video using MediaRecorder and get raw frames (byte arrays) from onPreviewFrame callback method

Seems it's not that easy, mb it's not even possible, I don't know...

But I found some answers (for similar questions) and people say that you should reconnect camera instance (Camera.reconnect()) after calling MediaRecorder.start() and set preview callback again

I tried something like this but it doesn't work (recording works but onPreviewFrame is never called)

I also tried to call Camera's stopPreview and startPreview methods after MediaRecorder.start() but seems we should not do it otherwise when I try to stop recording (MediaRecorder.stop()) after such actions app will stop responding (it becomes frozen)

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.sample_main);
    mPreview = findViewById(R.id.surface_view);
    new Handler().postDelayed(new Runnable() {
        @Override
        public void run() {
            testVideoRecording();
        }
    }, 1000);
}

@Override
public void onPreviewFrame(byte[] bytes, Camera camera) {
    Log.i(TAG, "onPreviewFrame");
}

private void testVideoRecording() {
    mCamera = Camera.open();

    Camera.Parameters parameters = mCamera.getParameters();
    List<Camera.Size> mSupportedPreviewSizes = parameters.getSupportedPreviewSizes();
    List<Camera.Size> mSupportedVideoSizes = parameters.getSupportedVideoSizes();
    Camera.Size optimalSize = CameraHelper.getOptimalVideoSize(mSupportedVideoSizes,
            mSupportedPreviewSizes, mPreview.getWidth(), mPreview.getHeight());

    CamcorderProfile profile = CamcorderProfile.get(CamcorderProfile.QUALITY_HIGH);
    profile.videoFrameWidth = optimalSize.width;
    profile.videoFrameHeight = optimalSize.height;

    parameters.setPreviewSize(profile.videoFrameWidth, profile.videoFrameHeight);
    mCamera.setParameters(parameters);
    try {
        mCamera.setPreviewTexture(mPreview.getSurfaceTexture());
    } catch (IOException e) {
        e.printStackTrace();
    }

    mCamera.setPreviewCallback(this);

    mMediaRecorder = new MediaRecorder();

    // Step 1: Unlock and set camera to MediaRecorder
    mCamera.unlock();
    mMediaRecorder.setCamera(mCamera);

    // Step 2: Set sources
    mMediaRecorder.setAudioSource(MediaRecorder.AudioSource.DEFAULT);
    mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.CAMERA);

    // Step 3: Set a CamcorderProfile (requires API Level 8 or higher)
    mMediaRecorder.setProfile(profile);

    // Step 4: Set output file
    mOutputFile = new File(Environment.getExternalStorageDirectory()
            + File.separator + "test.mp4");
    if (mOutputFile.exists()) mOutputFile.delete();

    mMediaRecorder.setOutputFile(mOutputFile.getPath());

    // Step 5: Prepare configured MediaRecorder
    try {
        mMediaRecorder.prepare();
    } catch (IOException e) {
        e.printStackTrace();
    }
    mMediaRecorder.start();

    // Step 6: try to set preview frame callback

    //mCamera.stopPreview();

    try {
        mCamera.reconnect();
    } catch (IOException e) {
        e.printStackTrace();
    }

    /*try {
        mCamera.setPreviewTexture(mPreview.getSurfaceTexture());
    } catch (IOException e) {
        e.printStackTrace();
    }*/

    mCamera.setPreviewCallback(this);

    //mCamera.startPreview();
}

So I want to know if it's even possible to use MediaRecorder and preview frame callback at the same time. If yes then how to do it properly?

Answer 1

(edited based on Alex's comment below)

Yes, it is possible in general. The camera API can be tricky, precision and persistence are needed.

First, start with example code that you know works. For example, the official tutorial on capturing video with original API is: https://developer.android.com/guide/topics/media/camera#capture-video . Great, now that you have example code working on your device, you are most of the way there. If you can't get it to work on your device, try on at least one other brand of device.

Next, make you own app code match the threading and API calls of the example. When you get an error or a freeze, look at logs to understand what happened. It is often something trivial like missing permissions in manifest.

Finally, logs, logs, logs. In this question, if you had posted errors from logs along with your code, you might have found a specific problem with a specific answer.

Good luck!

Answer 2

First of all, I strongly recommend to switch from the deprecated Camera API (Camera.open(), …) to the new camera2 API, unless your target devices are all below Android API 21. The new API is much more powerful and flexible. For example, it natively supports multiple targets in same CaptureSession (with limitations depending on CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL). Here is an example of using MediaRecorder and ImageReader in the same session.

When camera2 is at LEGACY level on the device, it may still be safer to use the old API directly (this is the native language of such cameras).

If you are stuck with the old API, consider one of the samples that record video using MediaCodec and MediaMuxer. It is more powerful than MediaRecorder, but requires more work.

The catch, obviously, is that MediaCodec appears at API 21, so these examples are mostly relevant for LEGACY devices.

If you really must work with old devices, you have no choice but run some alternative video encoder, fed from the frames that come to you in onPreviewFrame() callback.