Using a usb camera and a Raspbery Pi Zero 2W to create a fake stereo image

75 Views Asked by At

I'm pretty new at this but I have put quite a bit of work in to it and go nowhere so I am hoping one of you will be able and willing to help out.

I have built some 3D night vision goggles. I bought a small stereo camera and a single thermal camera. Both are USB. The screen in the goggles needs an HDMI input. I thought the easiest way round this would be to use a Raspberry Pi to take the USB connection and output HDMI.

As there have been no Raspberry PIs available for quite some time I bought a couple of Mango PIs and installed Android. There were Apps for what I needed - taking the stereo camera in as usb and passing it out to the screen as HTML. I also found an app that would take in the single thermal camera output and output a fake stereo image i.e. it doubled up the thermal camera to a "Google Cardboard" view. Problem is that frame rate and latency were bad and because Android is designed to use a touch screen I had to add a blue tooth joystick so that I could click OK to allow the App to access the USB ports every time it ran.

Raspberry Pi zero 2W became available so I bought a couple of them. For the stereo cam I installed the lite version of the operating system and used ffmpeg to pass the camera output through to the HDMI port. Works very well - good frame rate and fairly low latency.

Problem is how do I get the single thermal USB camera to appear in a Google cardboard format? All I need to do is to get the camera to appear as half size on the screen on one side and then duplicate that stream on the other side. This creates the image for each eye in the goggles.

I would have though it would be quite easy but it doesn't seem to be. Hstack wont work unless the two sources are different - resource not available. Tmux just creates extra terminal screens and doesn't duplicate anything.

Whatever the solution it needs to work with the lite operating system. The overhead of the GUI slows things down way to much in terms of latency and frames per second. There must be a split or duplicate command somewhere but I haven't found it yet.

Any help appreciated.

I tried every usb camera application that would run without a GUI

0

There are 0 best solutions below