open cv shows green screen on jetson nano

Question

My camera shows a green screen. I am using IMX 219 I don' know why the camera gives this output

import cv2
    
cap=cv2.VideoCapture(0)    
 
while True:
    r,im=cap.read()
    cv2.imshow('dd',im)
    k=cv2.waitKey(30) & 0xff
    if k==27:
         break
    
cap.release()
cv2.destroyAllWindows()

Answer 1

General theory

As said in this link, you can use v4l2-ctl to determine the camera capabilities. v4l2-ctl is in the v4l-utils:

$ sudo apt-get install v4l-utils

and then:

$ v4l2-ctl --list-formats-ext

Looking to the same link and to this other, I saw that you can also quickly test your camera launching:

# Simple Test
# Ctrl^C to exit
# sensor_id selects the camera slot: 0 or 1 on Jetson Nano B01
$ gst-launch-1.0 nvarguscamerasrc sensor_id=0 ! nvoverlaysink

This simple gst-launch example can be used to determine the camera modes that are reported by the sensor you are using. Say for example that you get this output:

GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000 

then you should adjust accordingly the next command:

$ gst-launch-1.0 nvarguscamerasrc sensor_id=1 ! \
   'video/x-raw(memory:NVMM),width=3264, height=2464, framerate=21/1, format=NV12' ! \
   nvvidconv flip-method=0 ! 'video/x-raw, width=816, height=616' ! \
   nvvidconv ! nvegltransform ! nveglglessink -e

sensor_id=1 represents the right CSI camera slot, it can be either 0 or 1. As you can see from this link, newer Jetson Nano Development Kits come with two CSI camera slots and you can use this attribute to specify the right one [0 is the default]. Please notice that at the same link they use sensor_mode instead of sensor_id, I'd try with both. You don't necessarily need to include flip-method which is documented here though. All of this should give you an idea for the values to be inserted in the code

Also, it has been noticed that the display transform is sensitive to width and height [in the above example, width=816, height=616]. If you experience issues, check to see if your display width and height is the same ratio as the camera frame size selected [in the above example, 816 and 616 are respectively a quarter of 3264 and 2464]

OpenCV

Looking around on nVidia forum I found this post. The solution in that case was to use:

cap = cv2.VideoCapture('nvarguscamerasrc ! video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)20/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink', cv2.CAP_GSTREAMER)

In your case though, the 20fps for your IMX 219 would be too high with a frame size equal to 3280x2464. As you can see from the first table of this link the suggested value is 15fps while here they suggest 21fps. I would suggest you to start with the width, height, framerate values retrieved in the previous section. A framerate value lower than the nominal one should help you test the connectivity

cap = cv2.VideoCapture('nvarguscamerasrc ! video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)15/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink', cv2.CAP_GSTREAMER)

A full sample where to include the previous line updated with the right values is available from here:

# MIT License
# Copyright (c) 2019 JetsonHacks
# See license
# Using a CSI camera (such as the Raspberry Pi Version 2) connected to a
# NVIDIA Jetson Nano Developer Kit using OpenCV
# Drivers for the camera and OpenCV are included in the base image

import cv2

# gstreamer_pipeline returns a GStreamer pipeline for capturing from the CSI camera
# Defaults to 1280x720 @ 60fps
# Flip the image by setting the flip_method (most common values: 0 and 2)
# display_width and display_height determine the size of the window on the screen


def gstreamer_pipeline(
    capture_width=1280,
    capture_height=720,
    display_width=1280,
    display_height=720,
    framerate=60,
    flip_method=0,
):
    return (
        "nvarguscamerasrc ! "
        "video/x-raw(memory:NVMM), "
        "width=(int)%d, height=(int)%d, "
        "format=(string)NV12, framerate=(fraction)%d/1 ! "
        "nvvidconv flip-method=%d ! "
        "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
        "videoconvert ! "
        "video/x-raw, format=(string)BGR ! appsink"
        % (
            capture_width,
            capture_height,
            framerate,
            flip_method,
            display_width,
            display_height,
        )
    )


def show_camera():
    # To flip the image, modify the flip_method parameter (0 and 2 are the most common)
    print(gstreamer_pipeline(flip_method=0))
    cap = cv2.VideoCapture(gstreamer_pipeline(flip_method=0), cv2.CAP_GSTREAMER)
    if cap.isOpened():
        window_handle = cv2.namedWindow("CSI Camera", cv2.WINDOW_AUTOSIZE)
        # Window
        while cv2.getWindowProperty("CSI Camera", 0) >= 0:
            ret_val, img = cap.read()
            cv2.imshow("CSI Camera", img)
            # This also acts as
            keyCode = cv2.waitKey(30) & 0xFF
            # Stop the program on the ESC key
            if keyCode == 27:
                break
        cap.release()
        cv2.destroyAllWindows()
    else:
        print("Unable to open camera")


if __name__ == "__main__":
    show_camera()

There is also a full snippet available at this other link that can help you find the reasons for failure

import cv2
import gi
gi.require_version('Gst', '1.0')
from gi.repository import Gst

def read_cam():
        cap = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")
        if cap.isOpened():
                cv2.namedWindow("demo", cv2.WINDOW_AUTOSIZE)
                while True:
                        ret_val, img = cap.read();
                        cv2.imshow('demo',img)
                        if cv2.waitKey(30) == ord('q'): 
                            break
        else:
                print ("camera open failed")

        cv2.destroyAllWindows()

if __name__ == '__main__':
        print(cv2.getBuildInformation())
        Gst.debug_set_active(True)
        Gst.debug_set_default_threshold(3)
        read_cam()

Lastly, if you can open the camera in GStreamer from the command line but not in Python have a check of the OpenCV version with the previous print(cv2.getBuildInformation()) or more shortly with:

print(cv2.__version__)

Starting with L4T 32.2.1 / JetPack 4.2.2, GStreamer support is built into OpenCV. The OpenCV version is 3.3.1 for those versions and if you are using earlier versions of OpenCV [most likely installed from the Ubuntu repository] you will get Unable to open camera errors