I'm trying to create a real-time wrapper around another application (steamcmd, a command line tool to download mods from the steam workshop).
When running steamcmd in its own, or through subprocess.Popen(args, stdout=None, creationflags=subprocess.CREATE_NEW_CONSOLE), the output appears in real time (eg. after each downloaded workshop item, it prints the ID of the item).
Yet when I create the process with subprocess.Popen(args, stdout=subprocess.PIPE, creationflags=subprocess.CREATE_NEW_CONSOLE) (or other creationflags), I only get big chunks at a time, with predictable delineations (it always hangs at the end of "Waiting for user info...", until all items are downloaded)
I'm using the following code to read from stdout:
while True:
out = process.stdout.read1()
print(out.decode("utf-8", errors="ignore"), end="", flush=True)
return_code = process.poll()
if return_code is not None:
break
I've also tried text mode with stdout.read(1) (read a single character at a time), or even stdout.readline (since the chunks are multiple lines) - but all have the same "chunking" behavior.
When I redirect the output of to a file and look at it with tail -f, it is similarly "chunked" - so it seems to be a general difference between "console output" and "pipe output".
How could I make Popen receive the output of steamcmd as if it was attached to a console window?
An example command line would be steamcmd.exe +login anonymous +workshop_download_item 281990 899653076 +workshop_download_item 281990 1596617351 +workshop_download_item 281990 1638580371 +workshop_download_item 281990 1724144458 (just some random publicly accessible workshop items items), and the tool can be found here: https://steamcdn-a.akamaihd.net/client/installer/steamcmd.zip