tqdm bar cannot update correctly when use pdsh

Question

I have code like this:

# tq_test.py
from tqdm.auto import tqdm
import time

if __name__ == "__main__":
    bar = tqdm(total=5)
    for _ in tqdm(range(3)):
        time.sleep(0.5)

    for _ in range(5):
        bar.update(1)

If I run it in local matchine /home/user/miniconda3/bin/python -u /home/user/python/tq_test.py, it is correct, two bar is updated correctly.

But when I run it with pdsh, the first bar is not updated correctly

pdsh -S -f 1024 -w mymachine /home/user/miniconda3/bin/python -u /home/user/python/tq_test.py

How can I fix this?

Answer 1

tqdm(position=1, disable=not is_main_process) solves this problem.

• Position=1 can ensure that PDSH prints a progress bar.
• not is_main_process can ensure only one process can print a progress bar, and this is a common trick used in tqdm.

The reason for this is that PDSH uses a line buffer, and it doesn't wrap when the program only prints a progress bar. Using position=1 allows you to use TQDM's own mechanism to get around this problem.