What is the color space produced by MSS sct.grab()?

Question

Python 3.10, opencv-python==4.5.4.60

I'm having a hard time understanding the color format of screenshots taken by MSS.

Here is the portion of my screen that I am screenshotting (with correct/expected colors, taken manually):

Here is the screenshot code I'm working with:

with mss.mss() as sct:
    monitor = rect
    res = np.array(sct.grab(monitor))[:, :, :3] # remove alpha

When I use cv2.imsave() or cv2.imshow(), the screenshot looks as expected. If I attempt to create masks on this image based on [R, G, B] colors, it produces opposite results, implying that the default color space of MSS Screenshots is BGRA.

If I do the following:

res = cv2.cvtColor(res, cv2.COLOR_BGRA2RGB)
cv2.imshow("res", res)
cv2.waitKey(0)

It produces this:

However, then I am able to perform RGB masking properly.

I may have just answered my own question, but is it correct to say that MSS Screenshots are in BGRA format, and OpenCV expects BGR format - which would explain why RGB images have reversed colors when using cv2.imshow() and cv2.imsave()?

Answer 1

You are right, MSS uses BGRA:

• https://python-mss.readthedocs.io/api.html?highlight=BGRA#mss.tools.mss.base.ScreenShot.__array_interface__
• https://github.com/BoboTiG/python-mss/blob/v7.0.1/mss/screenshot.py#L30
• https://github.com/BoboTiG/python-mss/blob/v7.0.1/mss/screenshot.py#L46-L47

You can even find some tips for the conversion: https://python-mss.readthedocs.io/examples.html#bgra-to-rgb

Indeed, I would be glad to enhance the documentation if you find something better :)

Answer 2

if you need it, here you go:

screenshot = sct.grab(bbox)
img = Image.frombytes('RGB', screenshot.size, screenshot.bgra, 'raw', 'BGRX')
arr = np.array(img)
res = cv2.cvtColor(arr, cv2.COLOR_BGRA2RGB)
out.write(res)