How to Draw Chinese Characters on a Picture with OpenCV

Question

Currently my project uses OpenCV to process pictures, draw some information and then push the video frames out through ffmpeg, because it is a live streaming form of pushing, the processing speed of a single frame is very high, the processing speed is slow, it will lead to push out the video stream dropped frames. After troubleshooting, the more time-consuming process is the process of drawing Chinese characters. Because OpenCV does not support drawing Chinese characters, I use Pillow to convert the image format every time I draw, and then convert the Pillow image back to numpy format after drawing. The following are the key code steps to draw Chinese characters:

cv2_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
pil_img = Image.fromarray(cv2_img)
draw = ImageDraw.Draw(pil_img)
font = ImageFont.truetype("simhei.ttf", 15, encoding="utf-8")
delta_y = 20
draw.text((x, y + delta_y), f"中文xxxxx", (255, 255, 255), font=font)
img = cv2.cvtColor(np.array(pil_img), cv2.COLOR_RGB2BGR)

The following solutions are known to work:

1. All drawing is done using Pillow (including drawing rectangular boxes, line segments, etc.)
2. Compile the OpenCV package that includes the freetype module. Compile OpenCV packages that include the freetype module. For the two solutions I proposed, it may not be very easy to deal with them due to some complicated reasons:
3. The drawing in the code is cross-drawing, OpenCV -> Pillow -> Opencv ...... The amount of changes would be larger if Pillow was used exclusively for drawing.
4. I am compiling OpenCV-python with freetype module, but even if it is compiled successfully, it will encounter some cross-platform porting, cross-version use and a series of problems, which can only be solved by recompiling, and the program is only used as a back-up program to The program is currently only used as an alternate solution.

Of course there are other solutions, if there is a program that meets my high processing speed requirements, welcome to the big brother to give reference.

Answer 1

There are several aspects to your question which I cannot make out, so here are some general thoughts:

• if performance is an issue, you must profile your code so that you don't waste your time optimising the wrong parts
• you don't necessarily have to convert your entire video frame to PIL.Image if you are only annotating one or two corners you could potentially pass them on their own
• you don't necessarily have to pass any of your OpenCV frame to PIL if you choose to annotate onto a, say, solid black background
• you don't have to create a new PIL.Image for every frame - you could create one at the start of your video then fill it with black at the start of each frame and then draw text onto it - you won't need to create a new drawing context then for each frame
• you don't need to convert BGR->RGB->BGR, you could just specify your PIL drawing in BGR colours
• you can just get the font once at the start of your video instead of for every frame