Why does PIL.ImageGrab.grab of specific window doesn't work properly?

Question

I am trying to capture specific windows, and displaying them using ImageGrab using the following code:

import cv2
import numpy as np
from PIL import ImageGrab
import pygetwindow
import pyautogui

titles = pygetwindow.getAllTitles()

while(1):
    cars = pygetwindow.getWindowsWithTitle('car - Google Search - Google Chrome')[0]
    print(cars.left, cars.top ,cars.width, cars.height)
    img = ImageGrab.grab(bbox=(int(cars.left),int(cars.top),cars.width,cars.height)) #x, y, w, h
    img_np = np.array(img)
    frame = cv2.cvtColor(img_np, cv2.CV_8U)
    cv2.waitKey(1)
    cv2.imshow("frame", frame)
cv2.destroyAllWindows()

It is somehow able to display Chrome, and able to follow the window, but I noticed that whether the x and y axis of the bbox is larger than the width and height, there will be an error. Furthermore, whenever I try to move the Chrome browser around, the frame's height and width key adjust itself. Any ideas on how to solve these issues?

Answer 1

The problem originates from your ImageGrab.grab call. Unfortunately, it's not mentioned in the documentation itself, but from the source code, you see, that it's:

ImageGrab.grab(bbox=(left, top, right, bottom))

So, you must calculate right and bottom accordingly. This code works perfectly fine for me:

import cv2
import numpy as np
from PIL import ImageGrab
import pygetwindow

titles = pygetwindow.getAllTitles()

while True:
    cars = pygetwindow.getWindowsWithTitle('car - Google Search - Google Chrome')[0]
    print(cars.left, cars.top, cars.width, cars.height)
    img = ImageGrab.grab(bbox=(int(cars.left),
                               int(cars.top),
                               int(cars.left + cars.width),
                               int(cars.top + cars.height)))
    frame = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
    cv2.waitKey(1)
    cv2.imshow('frame', frame)

Please notice, I also corrected your cv2.cvtColor call, such that proper colors are displayed.

----------------------------------------
System information
----------------------------------------
Platform:      Windows-10-10.0.19041-SP0
Python:        3.9.1
PyCharm:       2021.1.3
NumPy:         1.20.3
OpenCV:        4.5.2
Pillow:        8.3.0
pygetwindow:   0.0.9
----------------------------------------