I'm trying to pass Minio.get_object to subprocess.Popen, which calls age crypto. Unfortunatelly, it seems the content gets somehow mangled and thus age fails.

Can't understand what am I doing wrong. I have the following code:

with Minio(
    endpoint="endpoint",
    access_key="key",
    secret_key="secret",
).get_object(bucket_name="bucket", object_name="enc.py") as pipeIn:
    with subprocess.Popen(
        ["age", "--decrypt", "--identity", "./.age-ident", "--output", "-"],
        stdin=pipeIn,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
    ) as pipeCrypto:
        pipeIn.auto_close = True
        pipeIn.decode_content = False

        retCrypto = pipeCrypto.wait()

        if retCrypto != 0:
            print(pipeCrypto.stderr.read().decode())

And I get age: error: failed to read header: parsing age header: unexpected intro and a bunch of hex

Doing things manually mc cat bucket/path | age --decrypt --identity ./.age-ident --output - works fine. Using Popen.communicate also works fine:

with Minio(
    endpoint="endpoint",
    access_key="key",
    secret_key="secret",
).get_object(bucket_name="bucket", object_name="enc.py") as pipeIn:
    with subprocess.Popen(
        ["age", "--decrypt", "--identity", "./.age-ident", "--output", "-"],
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
    ) as pipeCrypto:
        pipeIn.auto_close = True
        pipeIn.decode_content = False

        cryptoOut, cryptoErr = pipeCrypto.communicate(pipeIn.read())

        print(cryptoOut.decode())

HTTPResponse has read method so it should be OK to pass it as stdin to Popen or so I though, unfortunately it doesn't seem to work.

P.S. I don't want to use communicate because

  1. I'll have to store content in memory and the file can be quite large
  2. I still need to pass output from age down to two more pipes, and so, those pipes need to use communicate too
0

There are 0 best solutions below