I have been using camera2 in an opengl application to write to a SurfaceTexture. Since this is going to put the image in the wrong format, I decided to use an ImageReader object to capture the camera output in : ImageFormat.YUV_420_888 as recommended by MLKit.
When I added this, things seem to work okay, but when the FPS of my application drops for a second, the camera stops updating (only happens when I use the ImageReader).
I have seen posts like : Camera2 ImageReader freezes repeating capture request
however, I have
ImageReader reader = ImageReader.newInstance(100, 100,
ImageFormat.YUV_420_888, 3);
reader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
Log.d("FINDME", " ON AVAILABLE STARTED");
Image image = null;
try {
image = reader.acquireNextImage();
//Do whatever you want with your Image
if (image != null) {
image.close();
}
} catch (IllegalStateException iae) {
if (image != null) {
image.close();
}
}
Log.d("FINDME", " ON AVAILABLE DONE");
}
}, handler2);
Surface previewSurface = new Surface(mSurfaceTexture);
mSurfaces.add(previewSurface);
mSurfaces.add(reader.getSurface());
mPreviewBuilder.addTarget(previewSurface);
mPreviewBuilder.addTarget(reader.getSurface());
mCameraDevice.createCaptureSession(mSurfaces, mCaptureSessionStateCallback,
mBackgroundHandler);
and doesn't seem to help. It does run fine if the FPS stays high, but if I try to swap to different rendering features (Shaders), the fps can drop for a split second, which freezes the camera. The app still runs fine, just the camera source no longer updates and never recovers. It isn't all shaders that cause this as well, only the ones that are more involved.
in logcat I see:
12-12 22:50:43.824 21329 23300 D FINDME : ON AVAILABLE DONE
12-12 22:50:43.854 21329 21371 I MediaRecorderJNI: finalize
12-12 22:50:43.854 21329 21371 I MediaRecorderJNI: release
12-12 22:50:43.854 1015 17804 D MPEG4Writer: Video track stopping. Stop source
12-12 22:50:43.854 1015 17804 E MPEG4Writer: Stop() called but track is not started
12-12 22:50:43.854 1015 17804 D MPEG4Writer: Audio track stopping. Stop source
12-12 22:50:43.854 1015 17804 E MPEG4Writer: Stop() called but track is not started
12-12 22:50:43.855 1015 22459 I ACodec : [OMX.SEC.naac.enc] Now Executing->Idle
12-12 22:50:43.855 1090 22460 I SEC_BASE_COMP: SEC_OMX_ComponentStateSet: Cur(3) -> Dest(2)
12-12 22:50:43.857 1015 22459 I ACodec : [OMX.SEC.naac.enc] Now Idle->Loaded
12-12 22:50:43.860 1090 22460 I SEC_BASE_COMP: SEC_OMX_ComponentStateSet: Cur(2) -> Dest(1)
12-12 22:50:43.861 1015 22459 I ACodec : [OMX.SEC.naac.enc] Now Loaded
12-12 22:50:43.861 1090 25268 I SEC_BASE_COMP: [OMX.SEC.naac.enc]: Normally terminated
12-12 22:50:43.862 1015 22459 I ACodec : [OMX.SEC.naac.enc] Now uninitialized
12-12 22:50:43.862 1015 22459 I ACodec : [] Now kWhatShutdownCompleted event : 8544
12-12 22:50:43.862 1015 22459 I MediaCodec: Codec shutdown complete
12-12 22:50:43.865 1015 17804 I StagefrightRecorder: property set(service.camera.rec.running, 0) result = 0
12-12 22:50:43.866 1015 22451 I ACodec : [OMX.qcom.video.encoder.avc] Now Executing->Idle
12-12 22:50:43.883 1015 22451 I ACodec : [OMX.qcom.video.encoder.avc] Now Idle->Loaded
12-12 22:50:43.883 1015 22451 I ACodec : [OMX.qcom.video.encoder.avc] Now Loaded
12-12 22:50:43.891 1090 5365 I OMX-VENC: Component Deinit
12-12 22:50:43.891 1090 5365 I OMX-VENC: Video encode perflock released
12-12 22:50:43.892 1015 22451 I ACodec : [OMX.qcom.video.encoder.avc] Now uninitialized
12-12 22:50:43.892 1015 22451 I ACodec : [] Now kWhatShutdownCompleted event : 8544
12-12 22:50:43.893 1015 22450 I MediaCodec: Codec shutdown complete
12-12 22:50:43.893 1015 22450 I ResourceManagerService: MediaInfo removed 2397600 (width 1080 height 2220) remained 15297120
12-12 22:50:43.893 1015 22450 I ResourceManagerService: getMediaResourceInfo resourceType : 2, size : 1
12-12 22:50:43.893 1015 22450 I ResourceManagerService: writeResourceInfo (PID : 21329, clientID : 3989906560, non-secure-codec/video-codec:1, 1080x2220(fps:30) - SoftCodec : No, Encorder : Yes)
12-12 22:50:43.893 28011 28024 V ResourceManagerHelper-JNI: JNIMediaResourceHelper::notify eventType : 2, ext1 : 0, ext2 : 0
12-12 22:50:43.893 28011 28024 V ResourceManagerHelper-JNI: notify eventType : 2, ext1 : 0, ext2 : 0
12-12 22:50:43.893 3381 29213 V ResourceManagerHelper-JNI: JNIMediaResourceHelper::notify eventType : 2, ext1 : 0, ext2 : 0
12-12 22:50:43.893 3381 29213 V ResourceManagerHelper-JNI: notify eventType : 2, ext1 : 0, ext2 : 0
12-12 22:50:43.893 28011 28024 I SemMediaResourceHelper: makeMediaResourceInfo mOwnResourceEventExcluded : false, mPid : 28011
12-12 22:50:43.893 28011 28024 I SemMediaResourceHelper: [1] makeMediaResourceInfo resourceType : 2 isSecured : false, pid : 21329, client id : 3989906560
12-12 22:50:43.893 3381 29213 I SemMediaResourceHelper: makeMediaResourceInfo mOwnResourceEventExcluded : false, mPid : 3381
12-12 22:50:43.893 3381 29213 I SemMediaResourceHelper: [1] makeMediaResourceInfo resourceType : 2 isSecured : false, pid : 21329, client id : 3989906560
12-12 22:50:43.893 28011 28047 I SemMediaResourceHelper: onRemove
12-12 22:50:43.893 3381 3381 I SemMediaResourceHelper: onRemove
12-12 22:50:43.893 28011 28047 I DrmMediaResourceHelper: onRemove size = 1
12-12 22:50:43.893 28011 28047 E DrmMediaResourceHelper: onRemove making Dpdrm to 0 second point
12-12 22:50:43.895 691 11810 V APM_AudioPolicyManager: releaseInput() 1630
12-12 22:50:43.895 691 11810 V APM_AudioPolicyManager: closeInput(1630)
12-12 22:50:43.896 28011 28047 E DrmManagerClient: DRM_DISPLAYPORT_DISABLE failed
12-12 22:50:43.896 1015 17804 V MediaPlayerService: Delete media recorder client
12-12 22:50:43.897 691 11810 D MultiRecordManager: closeInputStream() moduleHandle:10, modulename:primary, stream_in:0xeab9f0d0, source:0, dev:0xeab95fb0
12-12 22:50:43.897 691 11810 I MultiRecordManager: closeInputStream() skipMultiRecord
12-12 22:50:43.897 21329 21371 W System : A resource failed to call release.
12-12 22:50:43.898 691 11810 V APM_AudioPolicyManager: releaseInput() exit
12-12 22:50:43.898 21329 22463 E FINDME: VideoFragment: onClosed: android.hardware.camera2.impl.CameraCaptureSessionImpl@83f26d4
12-12 22:50:43.898 691 30849 V APM_AudioPolicyManager: listAudioPorts() role 0 type 0 num_ports 0 ports 0xeab97700
12-12 22:50:43.898 691 30849 V APM_AudioPolicyManager: listAudioPorts() got 0 ports needed 11
12-12 22:50:43.899 691 30849 V APM_AudioPolicyManager: listAudioPorts() role 0 type 0 num_ports 11 ports 0xe70d8d80
12-12 22:50:43.899 691 30849 V APM_AudioPolicyManager: listAudioPorts() got 11 ports needed 11
12-12 22:50:43.899 653 1131 D audio_hw_primary: adev_close_input_stream: enter:stream_handle(0xe885a500)
12-12 22:50:43.899 653 1131 V audio_hw_sec: sec_stop_voice_note_recording
12-12 22:50:43.899 653 1131 D audio_hw_primary: in_standby: enter: stream (0xe885a500) usecase(18: audio-record)
12-12 22:50:43.900 691 11810 V APM_AudioPolicyManager: listAudioPorts() role 0 type 0 num_ports 0 ports 0xeab976a8
12-12 22:50:43.900 691 11810 V APM_AudioPolicyManager: listAudioPorts() got 0 ports needed 11
12-12 22:50:43.900 691 11810 V APM_AudioPolicyManager: listAudioPorts() role 0 type 0 num_ports 11 ports 0xe7093580
12-12 22:50:43.900 691 11810 V APM_AudioPolicyManager: listAudioPorts() got 11 ports needed 11
12-12 22:50:43.908 21329 21376 E BufferQueueProducer: [ImageReader-1280x720f23m3-21329-3] queueBuffer: BufferQueue has been abandoned
12-12 22:50:43.908 982 23334 E Surface : queueBuffer: error queuing buffer to SurfaceTexture, -19
12-12 22:50:43.908 982 23334 E Camera3-OutputStream: returnBufferCheckedLocked: Stream 1: Error queueing buffer to native window: No such device (-19)
12-12 22:50:43.908 982 23334 E Camera3-Device: Can't return buffer to its stream: No such device (-19)
12-12 22:50:43.940 21329 21376 E BufferQueueProducer: [ImageReader-1280x720f23m3-21329-3] dequeueBuffer: BufferQueue has been abandoned
12-12 22:50:43.940 982 23322 E Camera3-OutputStream: getBufferLockedCommon: Stream 1: Can't dequeue next output buffer: No such device (-19)
12-12 22:50:43.940 982 23322 E Camera3-Device: RequestThread: Can't get output buffer, skipping request: No such device (-19)
12-12 22:50:43.949 21329 21376 E BufferQueueProducer: [ImageReader-1280x720f23m3-21329-3] queueBuffer: BufferQueue has been abandoned
12-12 22:50:43.950 982 23334 E Surface : queueBuffer: error queuing buffer to SurfaceTexture, -19