I am using YOLOv8 for segmentation of mitochondria in EM image stacks(a 3D volume cut up into 2D images). The regular segmentation model performs very well but I wanted to pair it with the object tracking capabilities for instance segmentation so we can potentially get counts of how many mitochondria are present in the volume.
Initially it looks like its working ok but on closer inspection I noticed that the segmentation masks generated via the model's track function were much worse. Running some tests I calculated that the masks generated via tracking were only picking up just under 80% of the mitochondria that the standard predict function's output was.
Furthermore, the output from first image run through the tracking model was always just as good as the normal segmentation and performance only dropped off later. So I discovered that without the persist=True flag the output was just as good as the normal segmentation. However, this of course defeats the whole purpose of using object tracking for instance segmentation.
I have not been able to find any resources online that can tell me what might be happening here. It seems very strange since predict and track both use the same underlying model(in Ultralytic's source code the track function even appears to be calling the predict function). Why wouldn't the tracking algorithm just label masks it isn't tracking as new objects instead of excluding them from the output altogether?
Here is my code for instance segmentation with object tracking
results = []
for img in images:
result = model.track(img, device='cpu' conf=0.4, retina_masks=True, persist=True)
results.append(result[0])
For normal segmentation the code is the same except the result line is:
result = model.predict(img, device='cpu', conf=0.4, retina_masks=True)