pyinstaller and upx on Linux - no difference in exe size

Question

I'm building a project with this script on Xubuntu 20.04.

pyinstaller --distpath $OUTPUT_DIR \
        --name $APP_NAME \
        --onefile \
        --hidden-import numpy \
        --hidden-import zmq \
        --hidden-import cv2 \
        --clean \
        --upx-dir "/home/developer/Downloads/upx-3.96-amd64_linux/" \
        --log-level DEBUG \
        main.py

Having the upx-dir option or not does not influence the size of the output file (in this case ~320MB). The same command run on Windows works fine. The debug output shows that the UPX module has been found:

623 INFO: PyInstaller: 4.2
623 INFO: Python: 3.8.5 (conda)
646 INFO: Platform: Linux-5.4.0-65-generic-x86_64-with-glibc2.10
646 INFO: wrote /home/developer/projects/build/linux_host/pyinstaller_build/MyProject.spec
646 DEBUG: Testing for UPX ...
686 INFO: UPX is available.

Any suggestions ?