How to use PTQ/QAT/INT8 in YOLO-NAS for object detection?

Question

I've custom-trained my model to PTQ and QAT with the tutorial

After this, I got some log files, .pth file, and ptq/qat onnx file from the output as in the tutorial. At the bottom of the tutorial, it says need to convert the qat-onnx file to an INT8 TensoRT file, then I converted it with the command trtexec --fp16 --int8 --onnx=model.onnx --saveEngine=model.trt

Now I got my TensorRT file (in a .trt format).

Then here goes my questions:

1. How to do object detection with a TensorRT file?
2. I've already got the .pth file from the output of PTQ & QAT training, why I can't directly use the .pth file for object detection? Just like the way I'm using the .pth file for object detection from a normal non-PTQ & QAT training, by using the code below:

from super_gradients.common.object_names import Models
from super_gradients.training import models

model = models.get(Models.YOLO_NAS_M,
checkpoint_path="yolonas-m/ckpt_best.pth",
num_classes=1)
predictions = model.predict("23.jpg")
predictions.show(show_confidence=False)

If I'm using the .pth file from the output of PTQ & QAT for object detection, it will have an error message as below:

ValueError                                Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/super_gradients/training/utils/checkpoint_utils.py](https://localhost:8080/#) in __call__(self, model_state_dict, checkpoint_state_dict)
198
199             if ckpt_val.shape != model_val.shape:
\--\> 200                 raise ValueError(f"ckpt layer {ckpt_key} with shape {ckpt_val.shape} does not match {model_key}" f" with shape {model_val.shape} in the model")
201             new_ckpt_dict\[model_key\] = ckpt_val
202         return new_ckpt_dict

ValueError: ckpt layer backbone.stem.conv.post_bn.weight with shape torch.Size(\[48\]) does not match backbone.stem.conv.branch_3x3.conv.weight with shape torch.Size(\[48, 3, 3, 3\]) in the model

3. If there's really no way to use the PTQ & QAT INT8 TensorRT file or .pth file, can I use the ONNX file for object detection that is generated in the process of PTQ & QAT? and how?

I'm a newbie discovering YOLO-NAS. Thanks, everyone.