pyds9: DS9 GUI Freezes After Displaying Two Large Images

Question

I've been given a short Python 2.7 script that watches for new FITS files dropping into a directory, using XPA to command an instance of DS9 8.0.1 (Fedora 30, xpa-devel.x86_64 2.1.18-7.f30) to display them w/in frames two at a time.

The original script uses a series of Popen() commands to launch xpaset to format the DS9 window and indicate what files to load. The code is ten years old, and unable to get through a night of observations without it and/or DS9 crashing/freezing, in part due to network delays affecting XPA.

I re-implemented the script from scratch using Python 3.7 and pyds9 (pyds9.noarch 1.8.1-3.2). The new code and DS9 will run all night when fed 6.5MB focusing images. But, after loading the first two full size 85MB images, DS9 locks up and requires a 'kill -9' to get rid of it.

The OS, xpa, and DS9 versions are the same, suggesting that the error lies within my script. A complication has been waiting for the process that writes the FITS files to finish and close the files.

In the interest of brevity, I'm hoping the reason for DS9 freezing with my use of pyds9 will be obvious from the trimmed down samples, below. Otherwise, I'll edit this to include functional code.

Edits: Addressing @Iguananaut's comments, I've consolidated the DS9 startup wait time, and added the new file search and the wait_til_file_is_stable function to the newer code sample.

The original Popen()-based instructions:

#!/bin/env python
from subprocess import Popen
from time import sleep
import uuid

new_fits_filename = ""
old_fits_filename = ""

display_id='This_Script-' + uuid.uuid4().hex[:6].upper()
ds9_pid = Popen(['ds9','-geometry','940x1200','-title',display_id]).pid
if Popen(['xpaget',display_id,'blink'],stdout=DEVNULL,stderr=DEVNULL).wait() == 0: 
    break
sleep(.5)
Popen(['xpaset','-p',display_id,'scale','mode','zscale'])
Popen(['xpaset','-p',display_id,'frame','1'])
Popen(['xpaset','-p',display_id,'cmap','cool'])
Popen(['xpaset','-p',display_id,'frame','2'])
Popen(['xpaset','-p',display_id,'cmap','bb'])
Popen(['xpaset','-p',display_id,'tile'])
while True:
    [detect new FITS file in NFS target directory]
    Popen(['xpaset','-p',display_id,'frame','1'])
    Popen(['xpaset','-p',display_id,'file','mosaicimage','wcs',directory+'/'+new_fits_filename])
    sleep(1)  # Sleep again to avoid DS9 crashing
    Popen(['xpaset','-p',display_id,'zoom','to','fit'])
    if old_fits_filename != "":
        Popen(['xpaset','-p',display_id,'frame','2'])
        Popen(['xpaset','-p',display_id ,'file','mosaicimage','wcs',directory+ '/'+ old_fits_filename])
        sleep(1)  # Sleep again to avoid DS9 crashing
        Popen(['xpaset','-p',display_id,'zoom','to','fit'])
    old_fits_filename = new_fits_filename

The new pyds9-based instructions:

#!/bin/env python3
import glob
import os
from pathlib import Path
import pyds9
from time import sleep
import uuid

def wait_til_file_is_stable(filename):
    the_file = Path(filename)
    the_current_size = 0
    the_previous_size = -1
    while the_current_size != the_previous_size:
        the_previous_size = the_current_size
        the_current_size = the_file.stat().st_size
        sleep(0.5)


global_d = None
fits_folder = Path("/data/fits/new/") # an NFS mount
fits_file_prefix = "image"
fits_file_pattern = fits_file_prefix + "*.fits"
fits_full_path = str(fits_folder.resolve() / fits_file_pattern)
new_fits_filename = ""
old_fits_filename = ""
wait_time_launch = 70

display_id = 'This_Script-' + uuid.uuid4().hex[:6].upper()
global_d = pyds9.DS9(display_id, start="-geometry 940x1200", wait=wait_time_launch, verify=True)
global_d.set('scale mode zscale')
global_d.set('frame 1')
global_d.set('cmap cool')
global_d.set('frame 2')
global_d.set('cmap bb')
global_d.set('tile')
while True:
    sleep(0.5)
    # detect new FITS file in NFS target directory
    dirlisting = glob.glob(fits_full_path)
    if len(dirlisting) > 0:
        new_fits_filename = max(filter(lambda fname: fits_file_prefix in fname, dirlisting), key=os.path.getctime)
    wait_til_file_is_stable(new_fits_filename)

    global_d.set('frame 1')
    global_d.set('file mosaicimage wcs ' + str(new_fits_filename))
    global_d.set('zoom to fit')
    if old_fits_filename != "":
        global_d.set('frame 2')
        global_d.set('file mosaicimage wcs ' + str(old_fits_filename))
        global_d.set('zoom to fit')
    old_fits_filename = new_fits_filename